hi, As the subject line suggests,if i have a binary executable which uses shared libraries, then the program while its being loaded will first map the libraries to the correct addressess and then execute the program.I was of the view that the linux dynamic loader ld-linux.so was responsible loads the shared libraries needed by a program and prepares the program to run.But I read one document on the net which said something else.It said that when we are using runtime linking(that is using dl_* calls) does the loader gets called, and for the normal cases its the the exec family of calls that actually makes the required arrangements.How far is this true. And as far as shared libraries are concerned,willa single copy of (first copy) always loaded in memory.If not then it must be getting loaded when there is a first reference to it.But what happens when all the references have gone, does it automatically get unloaded.Is there some difference between libc and other libraries, on when they get loaded.
> hi, > As the subject line suggests,if i have a binary executable which > uses shared libraries, then the program while its being loaded will > first map the libraries to the correct addressess and then execute the > program.I was of the view that the linux dynamic loader ld-linux.so > was responsible loads the shared libraries needed by a program and > prepares the program to run.But I read one document on the net which > said something else.It said that when we are using runtime > linking(that is using dl_* calls) does the loader gets called, and for > the normal cases its the the exec family of calls that actually makes > the required arrangements.How far is this true.
AFAIK the ld-linux.so file is mapped and called by the kernel itself during the execve call.
> And as far as shared libraries are > concerned,willa single copy of (first copy) always loaded in memory.If > not then it must be getting loaded when there is a first reference to > it.But what happens when all the references have gone, does it > automatically get unloaded.Is there some difference between libc and > other libraries, on when they get loaded.
Libraries (including ld-linux.so and glibc) are mmaped. That means each page is loaded when it is first accessed. And is cached in memory. With a large number of users glibc is likely to stay in physical memory until you shut down. There will be only one instance in memory of each page no matter how many users it has. (Except for some CoW pages.)
-- Kasper Dupont -- der bruger for meget tid på usenet. For sending spam use mailto:aaa...@daimi.au.dk It is NOT portable (Linus Benedict Torvalds 1991)
> program.I was of the view that the linux dynamic loader ld-linux.so > was responsible loads the shared libraries needed by a program and > prepares the program to run.
It is indeed.
> But I read one document on the net which > said something else.It said that when we are using runtime > linking(that is using dl_* calls) does the loader gets called, and for > the normal cases its the the exec family of calls that actually makes > the required arrangements.How far is this true.
I'm not completely sure what you're driving at here. When execve() is called, the kernel first maps the required executable to memory. It then determines if the executable is dynamically linked. If dynamic linking is required it maps in the dynamic loader (which can be specified to be anything by the executable, though it is normally ld-linux.so on Linux systems) and invokes its entry point passing information about the executable. If dynamic linking isn't required the entrypoint of the executable is invoked directly.
When the dynamic linker receives control it determines the needed libraries and maps them into memory as required. In addition the dl_* routines exist to provide a runtime link into the dynamic linker. The library in question, libdl.so, is really just a set of stubs that call on to the real ld-linux.so routines.
> And as far as shared libraries are > concerned,willa single copy of (first copy) always loaded in memory.If > not then it must be getting loaded when there is a first reference to > it.
I'm not clear what you're referring to. If you're asking if the dynamic linker maps them into memory when they're first referenced or straight away the answer is the later. That is, if an executable needs libc.so, libm.so and libtermcap.so, all of those libraries are mapped into memory before the executable begins execution.
If you're referring to the presence of the file in physical memory, the files are mmap()ed so only one copy of the read only areas would exist in memory at any time, read write pages would be copied on write.
You might like to play with the LD_DEBUG environment variable, the debugging information provided should make it much clearer how it all works.
> If you're referring to the presence of the file in physical memory, the > files are mmap()ed so only one copy of the read only areas would exist in > memory at any time, read write pages would be copied on write.
What i meant was,is libc or any other library gets loaded in physical memory,even before a program requires it.I mean does it get loaded by default even if noone has reffered to it till now or it gets loaded whenever the first program refers to it,because libc will be virtually used by everyone.Then is that the case for user created libraries or its different. I hope you are getting my question.If its loaded by default then it will use the memory unnessarily.
> > If you're referring to the presence of the file in physical memory, the > > files are mmap()ed so only one copy of the read only areas would exist in > > memory at any time, read write pages would be copied on write. > What i meant was,is libc or any other library gets loaded in physical > memory,even before a program requires it.I mean does it get loaded by
Nope, there are no 'magically' loaded libraries. While libc.so is indeed used by virtually every program it isn't mapped for the first time till the first program that uses it starts. As soon as no more processes have it mmaped it will cease to 'consume' memory.
> Nope, there are no 'magically' loaded libraries. While libc.so is indeed > used by virtually every program it isn't mapped for the first time till the > first program that uses it starts. As soon as no more processes have it > mmaped it will cease to 'consume' memory.
Not only that, but it's not even loaded then. It's mapped, but it won't be loaded until the memory it was mapped into is accessed! This is known as 'faulting in'.
When you execute a program on Linux, the program is mapped into memory and the 'next instruction to execute' pointer is pointer into that mapped memory. Each page is loaded from disk as it's accessed. So if you run a 240Mb executable, you won't see 240Mb of memory going away.
> > Nope, there are no 'magically' loaded libraries. While libc.so is indeed > > used by virtually every program it isn't mapped for the first time till > the > > first program that uses it starts. As soon as no more processes have it > > mmaped it will cease to 'consume' memory.
> Not only that, but it's not even loaded then. It's mapped, but it won't > be loaded until the memory it was mapped into is accessed! This is known as > 'faulting in'.
This seems similar to DLLs in windows,but i thought that dl_* calls were actually the equivalent of DLL's in Linux.Do you mean to say that the actual mapping is done at runtime when we use dl_* functions.Because as far as i know of dll's they are dynamically loaded and unloaded when the last reference to it caeses to exist.The unloading part is similar to unix shared libraries.Its the loading part, and dl_* calls are actually close/equal to the dll's loading.
> > > Nope, there are no 'magically' loaded libraries. While libc.so is indeed > > > used by virtually every program it isn't mapped for the first time till > > the > > > first program that uses it starts. As soon as no more processes have it > > > mmaped it will cease to 'consume' memory.
> > Not only that, but it's not even loaded then. It's mapped, but it won't > > be loaded until the memory it was mapped into is accessed! This is known as > > 'faulting in'.
> This seems similar to DLLs in windows,but i thought that dl_* calls > were actually the equivalent of DLL's in Linux.Do you mean to say that > the actual mapping is done at runtime when we use dl_* > functions.Because as far as i know of dll's they are dynamically > loaded and unloaded when the last reference to it caeses to exist.The > unloading part is similar to unix shared libraries.Its the loading > part, and dl_* calls are actually close/equal to the dll's loading.
You need to understand that there are two layers here, the ld.so handling and the kernel level handling. I've already explained in gross detail exactly how ld.so works and suggested you play with the LD_DEBUG environment variable to see it in action.
Below ld.so the kernel does it's magic. Which means that when ld.so asks the OS to map the file into memory, NOTHING actually happens until those pages are actually accessed. Futher, if those pages are unmodified or mapped read only, when the system is under pressure it may choose to throw them out of memory (since it knows it can always get them from the file as required). The presence or non presence of a mapped file in memory is entirely the kernel's decision.
In summary: - ld.so _maps_ the libraries specified as required by the executable into memory as soon as the program starts. - You can ask ld.so to map in further libraries or unmap existing libraries at run time using the dl* set of functions - The operating system chooses when the portions of the file should be loaded from disk into memory based, it does this when the pages are first needed. - This process is almost exactly how things work under Windows. i.e Windows also does the mapping at startup, can be asked to do more using LoadLibraryA() etc and the OS also chooses when to actually put the portions of the file into memory
Please tell us which bit is the bit you keep wondering about?
> > Not only that, but it's not even loaded then. It's mapped, but it won't > > be loaded until the memory it was mapped into is accessed! This is known as > > 'faulting in'. > This seems similar to DLLs in windows,but i thought that dl_* calls > were actually the equivalent of DLL's in Linux.
They are the logical equivalent in many respects, but they may differ in implementation details.
> Do you mean to say that > the actual mapping is done at runtime when we use dl_* > functions.
Yes, the file is mapped into memory by calling 'mmap'.
> Because as far as i know of dll's they are dynamically > loaded and unloaded when the last reference to it caeses to exist.
I don't think DLLs are loaded, in the sense that the code is copied from disk into physical memory, except by actual usage of the code/data in them.
> The > unloading part is similar to unix shared libraries. Its the loading > part, and dl_* calls are actually close/equal to the dll's loading.
I think you're using the term 'loading' in an imprecise way.
|
