I have software that's pretty fast, but I'd like to make it a little quicker. I've done as much to optimize the algorithm and data structures as can be done. I'm using a single thread, so there are a number of serial operations that could be run in parallel.

I'm thinking about spawning one thread per cpu core on the host machine, then setting their affinity so that each thread can only run on one core, and each core gets one thread. When a machine is somewhat idle, this should almost double the speed on my laptop, and do better on servers. There's a little bit of overhead managing the threads, and locking data, but the idea is probably a good one.

Unless some errant process monopolizes a particular core. Work seems to be pretty evenly distributed across both in my laptop, but ... I'm wondering if I'd be better off just letting Windows schedule the threads and hoping they really do run concurrently?

"There is no limit to improvements..."
The question is - what are you actually trying to achieve? Is it a practical goal or playing around on a specific computer, specific operating system, specific software development language, etc, etc...
If you want to make it running as fast as possible, why don't you run this 'thing' on a Japaneese weather forecast supercomputer?

PS. -Just to help maintaining this topic. It's late Friday in Melbourne, business week is over.

Originally Posted by ForrestCroce

I have software that's pretty fast, but I'd like to make it a little quicker. I've done as much to optimize the algorithm and data structures as can be done. I'm using a single thread, so there are a number of serial operations that could be run in parallel.

I'm thinking about spawning one thread per cpu core on the host machine, then setting their affinity so that each thread can only run on one core, and each core gets one thread. When a machine is somewhat idle, this should almost double the speed on my laptop, and do better on servers. There's a little bit of overhead managing the threads, and locking data, but the idea is probably a good one.

Unless some errant process monopolizes a particular core. Work seems to be pretty evenly distributed across both in my laptop, but ... I'm wondering if I'd be better off just letting Windows schedule the threads and hoping they really do run concurrently?

Yes, it is probably best to let the operating system handle thread scheduling. A major reason being that the OS is already handling threads for processes other than your own. So, it should be able to balance well (even if it is windows).

A major thing to keep track of are variables that are shared by instances of objects that run in different threads. Ideally you don't want them to change but, if they must make sure they are volatile.

The reason for this is, the values may be loaded in the cache of each CPU. If the value changes, then it may only be visible in the cache that changed it and main memory. But, the other CPU's cache may not see the change. So, ideally, don't change those types of values. Making the variables volatile, will ensure they always check against main memory, it'll also be slower as a result since, the cache has the same clock speed as the CPU and the main bus a fraction of that.

HTH