Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I was trying to optimize my Linux server to handle 10,000 threads per process while it does just 382 right now. As per this article the following formula is used to find out total possible threads:

number of threads = total virtual memory / (stack size*1024*1024)

This means threads store all their data in virtual memory. And to the best of my knowledge, virtual memory is swap space in a Linux machine which is stored on harddisk than RAM or cache.

So my question is does our threads uses harddisk to store for processing/store its data.

If yes, then doesnt this effect performance? Can we enhance the performance by putting in them on RAM or cache? How?

If no, how exactly do threads works?

Update:

According to useless's answer, virtual memory is a system comprising roughly:

  • physical memory (RAM)
  • any swapfiles you have attached
  • hardware support for translating virtual to physical addresses and issuing page faults when a virtual address isn't available in physical memory
  • (kernel) software support for: managing the lookup tables used by that hardware handling those page faults by pulling pages in from swap on demand

Thus, Everything that is on virtual memory is collectively on RAM(Real Memory) and Hard Disk(Swap Files). And as James explain in his answer decision on Ram vs HDD is taken by Kernel using algorithims such as LRU.

share|improve this question
1  
unless your server has 10,000 CPU/Cores you are wasting your time. –  Jarrod Roberson Feb 10 '12 at 14:33
    
@JarrodRoberson: Any why is that? –  dragosrsupercool Feb 10 '12 at 15:07
1  
10,000 threads isn't a good way to make things scale, it is a good way to make a server come to a crawl, more than 1 thread per CPU or Core is just going to make the server context switch and run slower not faster. –  Jarrod Roberson Feb 10 '12 at 15:45
    
Specifically, when you say "trying to optimize my Linux server" - what are you trying to optimize? If it's throughput, then one thread per CPU with multiplexing and non-blocking I/O is likely to be better. –  Useless Feb 10 '12 at 19:29

5 Answers 5

up vote 10 down vote accepted

to the best of my knowledge, virtual memory is swap space in a Linux machine

Nope, virtual memory is a system comprising roughly:

  • physical memory (RAM)
  • any swapfiles you have attached
  • hardware support for translating virtual to physical addresses and issuing page faults when a virtual address isn't available in physical memory
  • (kernel) software support for:
    • managing the lookup tables used by that hardware
    • handling those page faults by pulling pages in from swap on demand

It's up to the kernel to make sure the virtual memory you want is cached into RAM when you want it - unless you're writing your own userspace VM layer (such as databases often do, iiuc), just don't worry about it.

share|improve this answer
    
Ok so my virtual memory assumption was wrong. Anyways a quick followup question.. Would fully loaded max threads performance be affected if SWAP Space is more than RAM? –  dragosrsupercool Feb 10 '12 at 13:18
    
@dragosrsupercool: your swap space is always going to be bigger than physical memory, otherwise there's mo need to use virtual memory. –  Bryan Oakley Feb 10 '12 at 14:40
1  
@BryanOakley: That isn't necessarily true. Some OS's allocate a swap page for every virtual page allocated (i.e. swap must be at least as large as physical). Other OS's allocate a swap page only when there is a need to move a page out of physical memory (i.e. swap may be less than physical). The advantage of the former is that if the allocation succeeds, then swapping out that memory is always successful. The advantage of the latter is that you don't need to pessimistically allocate huge swap files to account for relatively rare situations. –  mcmcc Feb 10 '12 at 15:03
1  
@dragosrsupercool, performance won't be affected by the amount of RAM, swap or the ratio between them unless you're low on RAM and actually paging. sar can tell you about paging activity iirc (checked: sar -B on Linux). –  Useless Feb 10 '12 at 15:50
    
@Useless: I wish to increase the number of threads till I completely utilize RAM and don't start paging. –  dragosrsupercool Feb 10 '12 at 16:04

First of all, you need to read more on computer memory, because you seems to lack the knowledge in that field.

A thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process. Multiple threads can exist within the same process and share resources such as memory, while different processes do not share these resources.

So, threads are going to use available memory - whatever kind of it is available. How many threads you can start depends on the memory size and how much memory is needed per thread. If thread uses heap (not only stack), then it needs more memory, and in that case you can start less threads.

share|improve this answer
    
@VJonvic: +1 for basic thread explanation. –  dragosrsupercool Feb 10 '12 at 13:33

If the thread is actually running then the current instruction , and, any variables the thread is using must be in physical memory.

Most (in fact nearly all) programs reside in virtual memory, and, most programs use virtual memory for storage of variables.

Virtual addresses organized into chunks called pages (these are usually 4096 or 8192 byte blocks).

At any given time each block of virtual memory is stored somewhere in real memory or on the disk in the "swap space" reserved for this.

Your program code deals with virtual addresses, when you branch to a virtual address, or, request access to storage at a virtual address the system (usually at hardware level) locates the current location of the address request and maps it to your virtual address, if the address currently resides on the disk it pages it into real memory and then maps the address.

Obviously when all physical memory is in use if something is paged in then something else must be paged out, so the system looks for the "Least Recently Used" page and copies this out to disk before copying the page you requested in.

In modern systems there are several optimizations and tricks associated with virtual storage.

  • Addresses are mapped on a "per process" basis so for instance all C programs in a Linux box start the "main" process at the same address.
  • This can enable several 32 bit processes to occupy and use much more than 4GB on a machine as a 32 bit virtual address can be mapped to a real 64 bit address.
  • When processes end or memory is otherwise "free"ed the the system just marks the pages as free, they are never copied back to the swap disk.
  • Similarly when a new block of storage is requested the system just grabs a free page in real memory, no, disk IO takes place.
  • The sleep and hibernate functions force all the memory to be copied to the swap space so that all current processes and there current memory contents can be recreated on wake-up.
share|improve this answer
3  
The "all C programs in a Linux box start [main] at the same address" would seem to not take address space layout randomization into account. That's used more and more today to thwart various stack smashing attack schemes. Good answer otherwise, so +1. –  Michael Kjörling Feb 10 '12 at 12:53

The simple answer to your question is, they use virtual memory. everything uses virtual memory except a handful of processes related to the OS.

On the other hand, when your thread (or any thread, in any process) is actually running, it is using physical memory. The memory pages associated with that process are swapped in to physical memory which is where the processor does its work.

share|improve this answer

Virtual memory is your RAM plus your swap space. Virtual just means the address your program sees is different than the address the RAM chip sees. If you need to access memory in swap, the OS will move it into RAM first. If you don't want any swapping, just disable it. If you have enough RAM you don't really need it.

That being said, unless you have a 10,000 core processor, increasing to 10,000 threads isn't really an "optimization." Once you have enough threads to consume all the cores, plus a spare or two for when those threads are blocked, adding more threads decreases performance due to the switching overhead and cache misses. You might still want to use more threads if it makes your program logic simpler, but you will be trading off performance.

share|improve this answer
    
Yes, 10,000 is too much as my server is a 32bit single core machine. Actually, the threads are not total cpu thing. They are crawler threads, so they would be like waiting for server response sometimes. I aim to make sure cpu is totally occupied but not overload or underload. But I still fail to understand how can I know if CPU is like free or totally occupied. Is there any tool or command? –  dragosrsupercool Feb 10 '12 at 16:09
    
I think you can get that information from the top command. –  Karl Bielefeldt Feb 10 '12 at 16:19
    
@KarlBieledeldt: yes that was exactly what I was seeking for.. One more followup question: I just came with an idea for crawling that if some how one thread can send request for urls while the other thread receives the server response then I can keep CPU utilization high without using too many threads. Is that possible? Like sending request from one thread while receiving the response on the other thread? –  dragosrsupercool Feb 10 '12 at 16:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.