Author Topic: Slow scanning even with cache  (Read 104837 times)

DocMAX

  • Jr. Member
  • **
  • Posts: 7
    • View Profile
Re: Slow scanning even with cache
« Reply #345 on: October 17, 2013, 22:10:02 »
any news??? scanning takes whole days if > 3000!!!

Admin

  • Administrator
  • Hero Member
  • *****
  • Posts: 664
    • View Profile
    • https://www.smilarityapp.com
Re: Slow scanning even with cache
« Reply #346 on: November 04, 2013, 08:01:16 »
1. In you logs only 2 threads started do you have 2 cores CPU ?
2. What size of images/audio do you have (largest) ?
3. Try disable unncessary decoders (DirectShow, MMF, QuickTime, ACM)
4. You can disable fully image comparing algorithm just uncheck image content based algorithm and experemental in Options
4. Just scanned 13000 photo (up to 6Mb JPG) images, took 40 minutes on i7-2600

Springdream

  • Jr. Member
  • **
  • Posts: 51
    • View Profile
Re: Slow scanning even with cache
« Reply #347 on: October 15, 2014, 18:56:24 »
The slowing down also applies to me. First some 1000 are quite slow, then at let's say 10000 its 1 song per second.
I guess similarity has to compare each file against each other  meaning time effort is proportional to faculity of n (N!)?

I also have the feeling that the cache does not speed up too much.

ALSO another question on the cache:
1) does it slow down if there are too many item in the cache that do not exist anymore? AND
2) if so maybe a function like "delete cache items that do not exist anymore" would be nice
3) I have the feeling that the cache is relating to absolute paths of the files. Everytime there is a changed folder name all files below that folder are lost in the cache. Maybe the cache could refer to some other file properties like a hash, checksum ect. or name, size and date?

Admin

  • Administrator
  • Hero Member
  • *****
  • Posts: 664
    • View Profile
    • https://www.smilarityapp.com
Re: Slow scanning even with cache
« Reply #348 on: October 15, 2014, 20:37:03 »
Not N! it's N^2. N! very big values.
1) no, cache used as binary tree search time always log(N), it's only take time to load data and take RAM.
2) reasonable suggestion, we think about "purge cache" button.
3) yes cache uses absolutly pathes, because how we can found file equivalence ? For calculation hash or checksum we need disk access that takes time.

Igreenzevb

  • Jr. Member
  • **
  • Posts: 1
    • View Profile
    • Британские котята в дар.
Slow scanning even with cache
« Reply #349 on: March 07, 2016, 17:32:00 »
Hi,
I do not understand why clicking a tag in review mode is so slow, while clicking the same tag in the folder view shows me the actions in under a second.

e.g. I have one tag that has 19 items. In folder view 1 second in review mode it takes 14 seconds.

What I get to see is exactly the same data but why does it take such a different time to load the list?

Regards
Manu

Admin

  • Administrator
  • Hero Member
  • *****
  • Posts: 664
    • View Profile
    • https://www.smilarityapp.com
Re: Slow scanning even with cache
« Reply #350 on: March 07, 2016, 20:02:17 »
Hi,
I do not understand why clicking a tag in review mode is so slow, while clicking the same tag in the folder view shows me the actions in under a second.

e.g. I have one tag that has 19 items. In folder view 1 second in review mode it takes 14 seconds.

What I get to see is exactly the same data but why does it take such a different time to load the list?

Regards
Manu
What do you mean by review mode and folder view (first tab with folders doesn't show tags) ?

Springdream

  • Jr. Member
  • **
  • Posts: 51
    • View Profile
Re: Slow scanning even with cache
« Reply #351 on: September 19, 2018, 17:37:37 »
I guess you have simply too less RAM.
Consider 8GB+ for your collection...