04131355.txt 13-Apr-00 DLL Bloating Subject: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) Hi everyone, DLL's seem to be the way to go when you want to reuse your code and help easing upgrades. I've moved libraries to dll form to re-use their functionality but have found a weird behaviour. The environment is as follows: 1. There is a MainLib.dll that contains all extensions and special objects and routines I normally implement in my applications. 2. I have three app's that should share several files and objects, so the logic calls to use libraries or dll's with these. They were defined as Dll's from startup and intially used as libraries during the development and debug phase. Now I've changed them to full Dll funcionality: linking the dll and specifying the Dll.aef in the libraries for the app's. So far so good. 3. The hierarchy looks somewhat like this: App01 +----MainLib.Dll +----Servers.Dll | +----MainLib.Dll +----Objects.Dll +----MainLib.Dll +----Servers.Dll App02 +----MainLib.Dll +----Servers.Dll | +----MainLib.Dll +----Objects.Dll +----MainLib.Dll +----Servers.Dll I expected some drastical file size reduction in first place (this was not the main reason for Dll implementation), but found that Servers.Dll and Obejcts.Dll were really big. Lurking inside them with ShowDep.exe showed that they were 'exporting' not just what they have inside, but also their dependencies' contents. IOW, Servers.Dll only calls MainLib.Dll's dbserver extensions, but it also exports MainLib.Dll's DataWindow and everything else. Trying to asses if there was a problem, I deleted the reference to MainLib.Dll from Servers.Dll and hand coded the just the prototypes needed inside it. This time, Servers.Dll just exported its own functions and objects and the bunch of calls included in the prototypes. And also, the file size was reduced almost by 40%. What makes me curious about these is that none of the System Libraries prototypes were re-exported by my own dll's just the ones coming from my owns. So, here come the questions (still working with 2.0b4): 1. Is this normal behaviour? If not, what am I doing wrong? 2. Is there any way to prevent a Dll from export certaing functions and methods? In one of my libraries there is only need to export a class with its Init() and some Accesses and Assigns, but the Dll make everyghing visible. 3. Why if I specify the prototype library as a library (in a Dll), it pulls everything, not just what's needed? Any ideas out there? Many thanks in advance. Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: "Paul Piko" Amilcar, Sounds like you're linking the whole DLL code into the other DLLs rather than their prototypes. Take this example: Create a DLL application in VO, call it MainLib. Put in a function called MLDoIt(). Compile, build the DLL. Check with ShowDep - it exports MLDoIt(). Import the generated prototype library of MainLib into VO - the file will be called "MainLib - DLL.AEF", and the resulting library in VO will be called "MainLib DLL". Compile. Create another DLL application in VO, call it Servers. Put in a function called SDoIt(), and make it call MLDoIt(). Change the app properties of Servers to use the library "MainLib DLL". Compile, build DLL. Using ShowDep, you can see it exports SDoIt(). Now, if by mistake you put MainLib (the code that creates the MainLib.DLL) into the properties of Servers, then when you created Servers.DLL it will also export the MLDoIt() function. In effect what you end up doing is use the MainLib code as a VO library instead of as a DLL. So basically the rule is you _never_ link the DLL application code directly into anything. Whenever you create a DLL you have to import the generated prototype library and use it in your other apps/dlls. In answer to your other question, if you want to stop some entries in your DLL being exported you can use the STATIC modifier - e.g. STATIC FUNCTION, STATIC CLASS, STATIC METHOD - however this also restricts them to the module they're declared in. -- Paul Subject: Re: DLL Bloating From: "Paul Piko" Amilcar, The STATIC CLASS and METHOD were only for VO 2.0, not VO 2.5. Paul Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) On Thu, 16 Mar 2000 01:12:14 GMT, "Paul Piko" wrote: >Amilcar, > >The STATIC CLASS and METHOD were only for VO 2.0, not VO 2.5. Any workaround in VO 2.5 (I will upgrade by mid-year)? TIA, Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) On Wed, 15 Mar 2000 22:49:58 GMT, "Paul Piko" wrote: >Amilcar, > >Sounds like you're linking the whole DLL code into the other DLLs rather than their prototypes. Paul, No, I'm linking the prototypes AEF's. The whole process is: Start development of library in current proyect. When it's stable and (almost) bug-free, move it to another proyect, build the dll, import the Prototype Aef into original proyect and continue from there. I've no possibility of linking the whole library any more. [snip] >In answer to your other question, if you want to stop some entries in your DLL being exported you can use the STATIC modifier - e.g. STATIC FUNCTION, STATIC CLASS, STATIC METHOD - however this also restricts them to the module they're declared in. But they will be useless if needed across the Dll, not just in one module. It's a pitty. Thanks a lot, Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: Gryphon_Systems@telus.net (Ed Ratcliffe) Amilcar (look in your email) I use libraries for all of my common stuff like custom buttons, dialog popups etc. and dll's for app specific code. Also edit system generated import prototypes to only include what is needed. I have no experimental proof one way or the other but it just seemed to be right. Ed Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) Hi Ed, >Amilcar (look in your email) Thanks, I got it! >I use libraries for all of my common stuff like custom buttons, dialog popups etc. and dll's for app specific code. Also edit system generated import prototypes to only include what is needed. I have no experimental proof one way or the other but it just seemed to be right. Talking about editing prototypes. I've found another, let's call it bug: If you add something to your DLL, say another method or access or assign, the easiest way to get to it should be to edit into your DLL prototype in your proyect and let it go. I was doing this for some entities and suddenly appeared "not linked to" errors on start up of applications. The prototypes were fine, identical to what VO generated, but didn't work. Later on I've found that the linker creates the export table with "Proper" capitalization of the words "Access" and "Assign", but when you type them into the code editor, no matter how you try, they allways get converted to upper case. When you build your app it looks for ACCESS or ASSIGN, which are not in there. So your only way to "upgrade" your prototyes (AFAIK) is to re-import the whole proto library!. Regards, Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: "Paul Piko" Amilcar, Just turn off the Keyword Case option in the Editor Options to get around this. -- Paul Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) >Amilcar, > >Just turn off the Keyword Case option in the Editor Options to get around this. I knew there must be an easy way. Thanks a lot. Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: Larry Atkins Hi Amilcar, > 1. Is this normal behaviour? If not, what am I doing wrong? Unfortunately yes, this is normal VO behavior. The VO object model relies on the fact that the windows loader resolves the addresses of methods, accesses, and assigns via the import and export tables and so all method/access/assigns must be exported from a DLL. Furthermore, because VO is basically an untyped system, the linker cannot resolve which methods need to be imported from a DLL or library so it just imports everything. This "feature" is by far my biggest gripe with VO. If you use multiple DLLs in a hierarchical arrangement, you can easily create DLLs with 50% bloat. > 2. Is there any way to prevent a Dll from export certaing functions and methods? No and the static modifier has no effect on methods and classes. > Any ideas out there? Many thanks in advance. I wrote a little utility in C++ that helps with this problem. Basically it does 2 things. First it modifies the DLLs export table to export by ordinals instead of name. This can make the export table a lot smaller. The utility also bounces the export table against the AEF generated by the VO linker and only exports the entities that are truly in the DLL. As you alluded to, the linker exports classes from DLLs that DLL you are linking refers to. I believe this is a bug in the linker. I've uploaded the utility to our ftp site (ftp://www.dproinc.com/misc/VODLLDiet.EXE) feel free to download and experiment with it. For instructions, just run the utility and it prints out the command line arguments. Be aware that there is _absolutely_ no support for this utility. Use it at your own risk. BTW, I used it on the current ClassMate and ReportPro builds . -- Larry Atkins, DataPro Inc. Subject: Re: DLL Bloating From: "Geoff Schaller" Larry, Your VODLLDiet is both useful and instructive. Thankyou. I have often complained to myself that by splitting up my app into 5 dll's I was getting 5 export tables into the GUI library and that is some 320KB or so per dll. But I guess laziness is creeping in for me now. Although this is almost 1.5MB of wasted space it has bugger-all effect on load time because the whole thing is 20MB anyway. There are obviously savings to be made but with memory getting larger and cheaper by the day, maybe this issue is slowly dwindling in importance ? Geoff Subject: Re: DLL Bloating From: Larry Atkins Geoff, > Your VODLLDiet is both useful and instructive. Thankyou. My pleasure. > I have often complained to myself that by splitting up my app into 5 dll's I was getting 5 export tables into the GUI library and that is some 320KB or so per dll. But I guess laziness is creeping in for me now. Although this is almost 1.5MB of wasted space it has bugger-all effect on load time because the whole thing is 20MB anyway. You may be able to save more than 320KB per DLL. If you have 320KB in an export table, then everything that links against the DLL has 320KB in its import table. > There are obviously savings to be made but with memory getting larger and cheaper by the day, maybe this issue is slowly dwindling in importance ? Wow, that is the same response I got from Sabo . I guess it depends on your market. Try selling a 20 MB downloadable Internet ActiveX control. -- Larry Atkins, DataPro Inc. Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) Hi Larry, >Hi Amilcar, > >> 1. Is this normal behaviour? If not, what am I doing wrong? > >Unfortunately yes, this is normal VO behavior. The VO object model relies on the fact that the windows loader resolves the addresses of methods, accesses, and assigns via the import and export tables and so all method/access/assigns must be exported from a DLL. Furthermore, because VO is basically an untyped system, the linker cannot resolve which methods need to be imported from a DLL or library so it just imports everything. > >This "feature" is by far my biggest gripe with VO. If you use multiple DLLs in a hierarchical arrangement, you can easily create DLLs with 50% bloat. That's what I found! My idea is to build a hierarchical structure of DLL's so you can upgrade and mantain easilly. My big question is: Why this doesn't happen with the system libraries (those that come with VO). Just mine's are accumulating. [snip] >> Any ideas out there? Many thanks in advance. > >I wrote a little utility in C++ that helps with this problem. Basically it does 2 things. First it modifies the DLLs export table to export by ordinals instead of name. This can make the export table a lot smaller. The utility also bounces the export table against the AEF generated by the VO linker and only exports the entities that are truly in the DLL. As you alluded to, the linker exports classes from DLLs that DLL you are linking refers to. I believe this is a bug in the linker. > >I've uploaded the utility to our ftp site > >(ftp://www.dproinc.com/misc/VODLLDiet.EXE) > >feel free to download and experiment with it. For instructions, just run the utility and it prints out the command line arguments. Be aware that there is _absolutely_ no support for this utility. Use it at your own risk. BTW, I used it on the current ClassMate and ReportPro builds . Thanks a lot, I will download it ASAP and test. Regards, Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: Larry Atkins Hi Amilcar, > My big question is: Why this doesn't happen with the system libraries (those that come with VO). Just mine's are accumulating. I don't remember for VO 2.0, but in VO 2.5 the VO class libraries all export by ordinal. > Thanks a lot, I will download it ASAP and test. My pleasure. -- Larry Atkins, DataPro Inc. Subject: Re: DLL Bloating From: acamargo@gua.net (Amilcar Camargo) >Hi Amilcar, > >> My big question is: Why this doesn't happen with the system libraries (those that come with VO). Just mine's are accumulating. > >I don't remember for VO 2.0, but in VO 2.5 the VO class libraries all export by ordinal. I just looked inside CavoRt20.dll with ShowDep.exe and the export table has names in it, but the prototypes add the ordinal number. Maybe that makes a change. Regards, Amilcar A. Camargo Sand, S. A. Guatemala, C. A. Subject: Re: DLL Bloating From: "Phil McGuinness" snip [ (ftp://www.dproinc.com/misc/VODLLDiet.EXE) ] Love the name... Just a 1 calorie utility.. which should be good for 'fatties' out there. Hey can we run this on Windows 2000 and make it as skinny as Windows 3.11 Phil McGuinness Subject: Re: DLL Bloating From: Larry Atkins Phil, > Love the name... Thanks! Just hope Sabo has a sense of humor. > Hey can we run this on Windows 2000 and make it as skinny as Windows 3.11 Hmm...for some reason I don't think C++ has the same problem. -- Larry Atkins, DataPro Inc.