Slow scanning, crashes, etc

[pegreene]

When I first open the app it scans very quickly through the first 1000 or so files (in a group of 60-90k audio files). All of these files are cached at this point through days on end of running. Then it gets slower and slower to the point its less than 1 file every couple of seconds. System performance really gets drug down to crap as well. When the crashes happen the app continues to run but the commit memory usage is upwards of 40gb (I only have 12 gb system memory so the rest gets dumped in the page file. Ultimately have to close the program, wait a long while and eventually terminate the process to regain the memory. All of the log files look like this except the errors at the end repeat for what must be 100s of pages. The video driver has crashed a few times during use of the program as well, I do have the OpenCL processing turned on to aid with speed but it seems to make little difference and turning it off doesn't seem to make much difference in the scan speed either.

Intel Core i7 920 2.66GHz
12GB DDR3 16000 RAM
128GB Crucial M4 SSD System Drive
Western Digital WD2002FAEX 7200RPM 64MB Cache Storage Drive for the data being compared
Nvidia GeForce GTX 295 Video Card 896MB (335.23 Driver Version)
Windows 7 x64 Ultimate

Code: [Select]

2014-06-04 02:24:09
2014-06-04 02:24:09 Similarity version: 64-bit (x64) 1.9.1 build 1844 (4/10/2014)
2014-06-04 02:24:09 Windows version: 64-bit (x64) 6.1 SP1 (build 7601)
2014-06-04 02:24:09 Config init...
2014-06-04 02:24:09 Config init complete
2014-06-04 02:24:09 Module init...
2014-06-04 02:24:09 Module init complete
2014-06-04 02:24:09 Translator init...
2014-06-04 02:24:09 Translator init complete
2014-06-04 02:24:09 Start program
2014-06-04 02:24:09 State init...
2014-06-04 02:24:09 State init complete
2014-06-04 02:24:09 Menu creating...
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Thread started
2014-06-04 02:24:09 Menu creating complete
2014-06-04 02:24:09 Toolbar creating...
2014-06-04 02:24:09 Toolbar creating complete
2014-06-04 02:24:09 Statusbar creating...
2014-06-04 02:24:09 Statusbar creating complete
2014-06-04 02:24:09 Tabs creating(Folders)...
2014-06-04 02:24:09 OpenCL: NVIDIA CUDA - GeForce GTX 295
2014-06-04 02:24:09 OpenCL: GeForce GTX 295 version 335.23 (OpenCL 1.0 CUDA)
2014-06-04 02:24:09 OpenCL: Work group size 256
2014-06-04 02:24:09 OpenCL ready...
2014-06-04 02:24:09 Tabs creating(AudioResults)...
2014-06-04 02:24:09 Tabs creating(ImageResults)...
2014-06-04 02:24:09 Tabs creating(AudioAnalysis)...
2014-06-04 02:24:09 Tabs creating complete
2014-06-04 02:24:09 Adding tab pages complete
2014-06-04 02:24:09 Loading cache...
2014-06-04 02:24:12 Loading cache complete
2014-06-04 02:24:12 Loading ignores...
2014-06-04 02:24:12 Loading ignores complete
2014-06-04 02:24:12 Taskbar init complete
2014-06-04 02:24:12 Begin checking update
2014-06-04 02:24:12 Mainframe initialized...
2014-06-04 02:46:24 System error [203]: The system could not find the environment option that was entered. 
2014-06-04 02:46:24 COpenCL::Process:416 failed [false]
2014-06-04 02:46:24 System error [203]: The system could not find the environment option that was entered. 
2014-06-04 02:46:24 CAudioComparer::BlockCL:1035 failed [false]
2014-06-04 02:46:24 System error [203]: The system could not find the environment option that was entered. 
2014-06-04 02:46:24 CAudioComparer::ProcessCL:1116 failed [false]
2014-06-04 02:46:26 OpenCL use failed
2014-06-04 02:46:27 System error [203]: The system could not find the environment option that was entered. 


[pegreene]

After a system restart with opencl disabled its somewhat faster sometimes. Beyond that it gets back to 100% CPU and scanning only a 1-3 files per second usually. On first starting a scan it scans the first 3000 files in less than 3 minutes and gradually slows down from there. Taking hours on end to rescan 85000 files or so that it's already cached. Makes the process of cleaning a library rather tedious when it takes so long to rescan already processed files. I have the settings up pretty high so it's not like it's having to compare every files to every other file. Any ideas?

Going to restart everything before bed to start with a clean log while nothing else is going on. Seeing lots of this in the log currently and after closing the app there are a handful of decoder.exe processes left behind that never terminate by themselves.

Code: [Select]

2014-06-08 05:42:47 CProcessLink::Read:322 failed [false]
2014-06-08 05:42:47 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 05:42:47 CAudioDecoder::Open:59 failed [-3]
2014-06-08 05:42:50 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 05:42:50 CDecoderWMF::GetData:138 failed [-1]
2014-06-08 05:42:50 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 05:42:50 CAudioConverter::Decode:145 failed [false]
2014-06-08 05:42:50 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 05:42:50 CAudioComparer::LoadSound:82 failed [false]
2014-06-08 05:42:54 System error [2]: The system cannot find the file specified. 
2014-06-08 05:42:54 CProcessor::ProcessFolder:87 failed [false]
2014-06-08 05:42:59 System error [3]: The system cannot find the path specified. 
2014-06-08 05:42:59 CProcessor::ProcessFolder:87 failed [false]
2014-06-08 05:43:15 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 05:43:15 CDecoderWMF::GetData:138 failed [-1]
2014-06-08 05:43:15 System error [997]: Overlapped I/O operation is in progress. 


[Admin]

Sorry for this problems,
About OpenCL issue: Do you tried to benchmark OpenCL on Options before enabling, does it complete successful ?
What processes eats 40Gb, is it decoder.exe ? If so what means one of decoders is hang up/crashed and Similarity tries relaunch it. Because most time decoders execute 3rd party code very hard to find source of problem, please try to disable some decoders (it can be done in Options Decoders tabs) and enable them one by one, in your case disable WMF and DirectShow. Also you can uninstall 3rd party decoders.
If you want we can send you special version with extra logging to find source of problem, write to our email.

[pegreene]

It was the similiarity process that ate up the memory. The decoders are usually pretty low.

Benchmark is success ful on sizes work group sizes 1-256, Error on 512 and 1024. CPU Speed (1 core) 561ms. OpenCL device speed gets pretty high(1000+ms) on 32 or less size, pretty consistent 64-256 around 315ms.

Definitely performing better with it turned off which seems wrong. Still a fair bit of this type stuff in the logs. Just wish it would 'refresh' scan a little faster. After deleting, renaming, or retagging a few hundred files I wish it didn't have to start from scratch to pick up the changes.

Code: [Select]

2014-06-08 08:15:13 Decoder process is hang up
2014-06-08 08:15:13 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 08:15:13 CProcessLink::Read:322 failed [false]
2014-06-08 08:15:13 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 08:15:13 CAudioDecoder::Open:59 failed [-3]
2014-06-08 08:15:13 Decoder process is hang up
2014-06-08 08:15:13 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 08:15:13 CProcessLink::Read:322 failed [false]
2014-06-08 08:15:13 System error [997]: Overlapped I/O operation is in progress. 
2014-06-08 08:15:13 CAudioDecoder::Open:59 failed [-3]

[Springdream]

The initial effect of slowing down also applies to me using 191 192 beta...
Any solution?

[Springdream]

could it be that the algorithm to compare the songs is slowing down the progress and not the calculation of the fingerprint? also for >100000 songs in the cache the cache seems to get ineffective?!

[Admin]

Yes, for 100K files main problem is algorithm complexity, for Similarity it's N^2. That means for 1000 files it takes 1000^2 = 10^6 ephemeral units of time, 100000 files it takes 100000^2 = 10^10 ephemeral units of time, difference 10000 times, not 100 as for linear complexity. Very big speed improvement can be achieved by duration check, it skips many unnecessary comparisons, but complexity still N^2.
We working right now on 4th algorithm (in truth 5th, Similarity already have 4 audio algorithms, 2 of them just mixed by name "precise" one for long songs, one for short), it must have linear complexity, but work still in progress, we don't think we can release it in next version.
In current versions better solution to limit comparison areas by limiting selected folders or using folder groups and cleaning/moving cache.