0002982: performance issue with lots of regions (bad scalability)

(0008082)
confusion_music (reporter)
2010-05-27 13:55

For what its worth I find this to be quite an issue as well with a similarly specified PC as realhangman.

(0008084)
paul (administrator)
2010-05-27 14:30

this has already been addressed in ardour3. the entire undo/redo mechanism has been reimplemented for everything except automation data, which is much more compact than region state.

(0010549)
realhangman (reporter)
2011-04-12 09:20

Hi, an update: This has nothing to do with undo/redo. True, undo/redo gets very slow in A2 in a big session, this has been adressed in A3.

But my problem is the following scenario:

In a recent project (with A2 rev 9332), I have hundreds of regions and reduced the undo history to 7 and have no saved undo region at all (this gets saved now). Immediately after reopening the project, the gui (zooming, scrolling etc.) is very very slow. On top of that, Ardour crashes often while editing (splitting, trimming).

If I consolidate most of the regions, A2 is fast again, so it really is an issue with lots of (overlapping ?) regions.

I'd like to upload the project, but it's 20 GB big...

(0010550)
realhangman (reporter)
2011-04-12 09:30

Fwiw, I attach a zip file containing two session files.

The first file (2011-04-12T13:21:50.ardour) contains my slow session with hundreds of overlapping regions in the project.
The second (A2mainsession.ardour) contains the session with consolidated regions, i.e. no regions are deleted from the project but most have been removed from the timeline. A2 is quite fast here. This file contains some more data as it contains 2 more recorded songs.

Don't know if this is useful, tell me if you need something more.

(0010554)
realhangman (reporter)
2011-04-14 15:03

Here we go with a video & detailed description how to reproduce this fast.

Link: http://www.septimbears.de/a2-performance.ogv [^]

This is what I do:

1. Open a new project and import a file (this is a .flac file with about 5 minutes length)

--> Zooming and scrolling is fast

2. Split the region and copy one region over the other (Ctrl-left mouse and drag) some times (maybe 10-20) so we end up with quite some regions and crossfades.

--> Zooming and scrolling is already lagging

3. Do it some more

--> In this example, Ardour freezes for some time and is afterwards really slow. I never experienced this freeze and 100% CPU consumption before, but zooming and scrolling is painfully slow now.

4. Delete the regions

--> A2 is performing nicely again.

Conclusion: The undo/redo mechanism seems not to be the problem here. The problem is caused by too many overlapping regions, maybe also the underlying created crossfades. I have the feeling that it's better if you deactivate "create crossfades" beforehand, but it's not really that much better.
I know this is not a real world example, but imaging you have 14 tracks for drums and then cut and overlap some takes etc. You'll reach the limit very soon.

(0010555)
oofus (developer)
2011-04-14 15:31

maybe this is a flac issue. can you try the same thing with a wav file ?

(0010560)
realhangman (reporter)
2011-04-15 00:58

Thanks for replying. Unfortunately, the behaviour is completely the same with a 16 bit wav file. You can also record a short region and then follow the same procedure as described above.

(0010561)
lincoln (reporter)
2011-04-15 05:16

Have you tried to open this session on A3. I would be interested to see if there is any noticeable difference in performance.

(0010582)
realhangman (reporter)
2011-04-18 14:32

A3 feels very slightly better, but has the same problem.
I realized something else, though: Ardour only slows down as long as it displays the regions. If you are scrolling hoizontally to another part of the project with little or no regions, the gui behaves normal. The vertical zoom level of the track is also important, i.e. the smaller the track height the better the performance.

(0012396)
farbro (reporter)
2011-12-12 15:27

I confirm this issue on latest SVN. I do this kind of songs when I end up with lots and lots of short regions with samples. When reaching a somewhat large number of regions, the GUI starts performing extremely slow, especially copying/moving multiple regions completely locks up the computer and you have to leave it for up to 5 minutes before it's done processing the copying and you can start working again (though still with bad performance in scrolling etc).

(0012542)
lievenmoors (reporter)
2012-01-10 15:54

I would like to confirm this issue as well, at least for the ardour2 series.
Could this be due to the calculation or rendering of the waveforms?

(0012990)
realhangman (reporter)
2012-03-20 04:22

A quick note: I also experience the problem of slow scrolling within Audacity. When I have long file (about 30 Minutes) and zoom to near-sample level, horizontal scrolling becomes sluggish. I don't know if it's related, a shared library comes to my mind, but it also might be a coincidence.

(0013178)
farbro (reporter)
2012-04-24 05:58

I have now been running Ardour SVN on my laptop since January and I must say that I do no longer experience these performance issues. It is still a bit slow but much better than on my stationary computer.

My laptop has an Intel Core i5 2410M and 4GB RAM and is running Ubuntu 11.10. The stationary has an Intel Core 2 Duo E8400 and 4 GB RAM on 11.04.

Any clue of what is causing the problem?

(0013530)
cth103 (administrator)
2012-06-14 03:15

Are we thinking that A3 is ok in this regard now?

(0013532)
farbro (reporter)
2012-06-14 05:21

After further testing on both computers, I must now withdraw my last post. I haven't really experienced this problem with my laptop on my new projects, probably because they haven't been that complex. But now when I open up my old sessions on both of the computers, it performs just as bad on both.

This is my procedure to reproduce it:

1. Import a short half beat sample to say 5 tracks.
2. Duplicate them 20 times. Select them all and duplicate them, it goes pretty smooth.
3. Now, undo and duplicate the regions to a length of 100, this takes a moment but not too long.
4. Try duplicating all of them again, this will take a while and my Ubuntu freezes until it's done.

5. Delete all regions except the first ones. Do step 2 again, but this time the duplication will take *much* longer than the first time.

Apparently the bad performance starts when you reach a great amount of regions, and does not go away simply because you delete them.

This is on SVN 12717

(0013563)
realhangman (reporter)
2012-06-17 14:06

A3 rev.12747 is much better than A2 here! It still becomes slower with lots of overlapping regions, but I think it's much better. I still need to do some tests, though. I will report back.

(0013567)
cth103 (administrator)
2012-06-18 09:58

Should be quicker in SVN 12753.

(0013569)
farbro (reporter)
2012-06-18 10:57

Can't see any difference here... :/

(0013570)
cth103 (administrator)
2012-06-18 11:18

What are you testing? I meant the duplicating regions test.

(0013572)
realhangman (reporter)
2012-06-18 12:04
edited on: 2012-06-18 12:05

The duplications run fine here with A3 12756.

The performance with lots of regions though.. it's better and I don't expect a program to run perfectly smooth when stuffed with thousands of objects, but you can already feel this sluggishness after duplicating 5 regions 20 times, and after duplicating them a hundred times ( =500 regions), you'll see it for sure.

Again, this seems like a high region count, but when recording drums or whole bands with 10-20 tracks at once, you can't have too many takes before things will get slow.

(0013574)
cth103 (administrator)
2012-06-18 16:06

Is this sluggishness in moving around the session?

(0013593)
realhangman (reporter)
2012-06-19 03:30

Yes, when moving and zooming. Or when dragging a lot of regions, single regions are fine.

Issue History
Date Modified	Username	Field	Change
2010-01-01 10:44	realhangman	New Issue
2010-01-01 10:44	realhangman	Sponsorship Added	realhangman: US$ 50
2010-01-01 10:44	realhangman	Sponsorship Total	0 => 50
2010-01-01 10:44	realhangman	Issue Monitored: realhangman
2010-04-24 03:28	cth103	Category	bugs => bugs2
2010-04-24 03:30	cth103	Category	bugs2 => bugs
2010-05-27 13:55	confusion_music	Note Added: 0008082
2010-05-27 13:56	confusion_music	Issue Monitored: confusion_music
2010-05-27 14:30	paul	Note Added: 0008084
2011-04-12 09:20	realhangman	Note Added: 0010549
2011-04-12 09:30	realhangman	Note Added: 0010550
2011-04-12 09:31	realhangman	File Added: slow-project.zip
2011-04-14 14:05	realhangman	Sponsorship Updated	realhangman: US$ 100
2011-04-14 14:05	realhangman	Sponsorship Total	50 => 100
2011-04-14 15:03	realhangman	Note Added: 0010554
2011-04-14 15:31	oofus	Note Added: 0010555
2011-04-15 00:58	realhangman	Note Added: 0010560
2011-04-15 02:46	confusion_music	Issue End Monitor: confusion_music
2011-04-15 05:16	lincoln	Note Added: 0010561
2011-04-18 14:32	realhangman	Note Added: 0010582
2011-12-12 15:27	farbro	Note Added: 0012396
2012-01-10 15:54	lievenmoors	Note Added: 0012542
2012-01-11 04:26	farbro	Sponsorship Added	farbro: US$ 40
2012-01-11 04:26	farbro	Sponsorship Total	100 => 140
2012-01-11 04:26	farbro	Issue Monitored: farbro
2012-03-20 04:22	realhangman	Note Added: 0012990
2012-04-24 05:58	farbro	Note Added: 0013178
2012-04-25 07:33	cth103	cost	=> 0.00
2012-04-25 07:33	cth103	Target Version	=> 3.0
2012-06-14 03:15	cth103	Note Added: 0013530
2012-06-14 03:15	cth103	Status	new => feedback
2012-06-14 05:21	farbro	Note Added: 0013532
2012-06-14 16:33	cth103	Severity	major => block
2012-06-14 16:33	cth103	Status	feedback => acknowledged
2012-06-17 14:06	realhangman	Note Added: 0013563
2012-06-18 09:58	cth103	Note Added: 0013567
2012-06-18 10:57	farbro	Note Added: 0013569
2012-06-18 11:18	cth103	Note Added: 0013570
2012-06-18 12:04	realhangman	Note Added: 0013572
2012-06-18 12:05	realhangman	Note Edited: 0013572
2012-06-18 16:06	cth103	Note Added: 0013574
2012-06-19 03:30	realhangman	Note Added: 0013593

	Users sponsoring this issue
	Sponsors List	Total Sponsorship = US$ 140 2010-01-01 10:44: realhangman (US$ 100) 2012-01-11 04:26: farbro (US$ 40)

Notes
(0008082) confusion_music (reporter) 2010-05-27 13:55	For what its worth I find this to be quite an issue as well with a similarly specified PC as realhangman.

(0008084) paul (administrator) 2010-05-27 14:30	this has already been addressed in ardour3. the entire undo/redo mechanism has been reimplemented for everything except automation data, which is much more compact than region state.

(0010549) realhangman (reporter) 2011-04-12 09:20	Hi, an update: This has nothing to do with undo/redo. True, undo/redo gets very slow in A2 in a big session, this has been adressed in A3. But my problem is the following scenario: In a recent project (with A2 rev 9332), I have hundreds of regions and reduced the undo history to 7 and have no saved undo region at all (this gets saved now). Immediately after reopening the project, the gui (zooming, scrolling etc.) is very very slow. On top of that, Ardour crashes often while editing (splitting, trimming). If I consolidate most of the regions, A2 is fast again, so it really is an issue with lots of (overlapping ?) regions. I'd like to upload the project, but it's 20 GB big...

(0010550) realhangman (reporter) 2011-04-12 09:30	Fwiw, I attach a zip file containing two session files. The first file (2011-04-12T13:21:50.ardour) contains my slow session with hundreds of overlapping regions in the project. The second (A2mainsession.ardour) contains the session with consolidated regions, i.e. no regions are deleted from the project but most have been removed from the timeline. A2 is quite fast here. This file contains some more data as it contains 2 more recorded songs. Don't know if this is useful, tell me if you need something more.

(0010554) realhangman (reporter) 2011-04-14 15:03	Here we go with a video & detailed description how to reproduce this fast. Link: http://www.septimbears.de/a2-performance.ogv [^] This is what I do: 1. Open a new project and import a file (this is a .flac file with about 5 minutes length) --> Zooming and scrolling is fast 2. Split the region and copy one region over the other (Ctrl-left mouse and drag) some times (maybe 10-20) so we end up with quite some regions and crossfades. --> Zooming and scrolling is already lagging 3. Do it some more --> In this example, Ardour freezes for some time and is afterwards really slow. I never experienced this freeze and 100% CPU consumption before, but zooming and scrolling is painfully slow now. 4. Delete the regions --> A2 is performing nicely again. Conclusion: The undo/redo mechanism seems not to be the problem here. The problem is caused by too many overlapping regions, maybe also the underlying created crossfades. I have the feeling that it's better if you deactivate "create crossfades" beforehand, but it's not really that much better. I know this is not a real world example, but imaging you have 14 tracks for drums and then cut and overlap some takes etc. You'll reach the limit very soon.

(0010555) oofus (developer) 2011-04-14 15:31	maybe this is a flac issue. can you try the same thing with a wav file ?

(0010560) realhangman (reporter) 2011-04-15 00:58	Thanks for replying. Unfortunately, the behaviour is completely the same with a 16 bit wav file. You can also record a short region and then follow the same procedure as described above.

(0010561) lincoln (reporter) 2011-04-15 05:16	Have you tried to open this session on A3. I would be interested to see if there is any noticeable difference in performance.

(0010582) realhangman (reporter) 2011-04-18 14:32	A3 feels very slightly better, but has the same problem. I realized something else, though: Ardour only slows down as long as it displays the regions. If you are scrolling hoizontally to another part of the project with little or no regions, the gui behaves normal. The vertical zoom level of the track is also important, i.e. the smaller the track height the better the performance.

(0012396) farbro (reporter) 2011-12-12 15:27	I confirm this issue on latest SVN. I do this kind of songs when I end up with lots and lots of short regions with samples. When reaching a somewhat large number of regions, the GUI starts performing extremely slow, especially copying/moving multiple regions completely locks up the computer and you have to leave it for up to 5 minutes before it's done processing the copying and you can start working again (though still with bad performance in scrolling etc).

(0012542) lievenmoors (reporter) 2012-01-10 15:54	I would like to confirm this issue as well, at least for the ardour2 series. Could this be due to the calculation or rendering of the waveforms?

(0012990) realhangman (reporter) 2012-03-20 04:22	A quick note: I also experience the problem of slow scrolling within Audacity. When I have long file (about 30 Minutes) and zoom to near-sample level, horizontal scrolling becomes sluggish. I don't know if it's related, a shared library comes to my mind, but it also might be a coincidence.

(0013178) farbro (reporter) 2012-04-24 05:58	I have now been running Ardour SVN on my laptop since January and I must say that I do no longer experience these performance issues. It is still a bit slow but much better than on my stationary computer. My laptop has an Intel Core i5 2410M and 4GB RAM and is running Ubuntu 11.10. The stationary has an Intel Core 2 Duo E8400 and 4 GB RAM on 11.04. Any clue of what is causing the problem?

(0013530) cth103 (administrator) 2012-06-14 03:15	Are we thinking that A3 is ok in this regard now?

(0013532) farbro (reporter) 2012-06-14 05:21	After further testing on both computers, I must now withdraw my last post. I haven't really experienced this problem with my laptop on my new projects, probably because they haven't been that complex. But now when I open up my old sessions on both of the computers, it performs just as bad on both. This is my procedure to reproduce it: 1. Import a short half beat sample to say 5 tracks. 2. Duplicate them 20 times. Select them all and duplicate them, it goes pretty smooth. 3. Now, undo and duplicate the regions to a length of 100, this takes a moment but not too long. 4. Try duplicating all of them again, this will take a while and my Ubuntu freezes until it's done. 5. Delete all regions except the first ones. Do step 2 again, but this time the duplication will take much longer than the first time. Apparently the bad performance starts when you reach a great amount of regions, and does not go away simply because you delete them. This is on SVN 12717

(0013563) realhangman (reporter) 2012-06-17 14:06	A3 rev.12747 is much better than A2 here! It still becomes slower with lots of overlapping regions, but I think it's much better. I still need to do some tests, though. I will report back.

(0013567) cth103 (administrator) 2012-06-18 09:58	Should be quicker in SVN 12753.

(0013569) farbro (reporter) 2012-06-18 10:57	Can't see any difference here... :/

(0013570) cth103 (administrator) 2012-06-18 11:18	What are you testing? I meant the duplicating regions test.

(0013572) realhangman (reporter) 2012-06-18 12:04 edited on: 2012-06-18 12:05	The duplications run fine here with A3 12756. The performance with lots of regions though.. it's better and I don't expect a program to run perfectly smooth when stuffed with thousands of objects, but you can already feel this sluggishness after duplicating 5 regions 20 times, and after duplicating them a hundred times ( =500 regions), you'll see it for sure. Again, this seems like a high region count, but when recording drums or whole bands with 10-20 tracks at once, you can't have too many takes before things will get slow.

(0013574) cth103 (administrator) 2012-06-18 16:06	Is this sluggishness in moving around the session?

(0013593) realhangman (reporter) 2012-06-19 03:30	Yes, when moving and zooming. Or when dragging a lot of regions, single regions are fine.

Relationships