From: David Laight To: tech-kern@netbsd.org Subject: link-sets in modules Date: Mon, 28 May 2012 07:46:12 +0100 There are currently problems loading modules that contain link-sets (link-sets were the 'flavour of the month' a while back!) because the link sets don't get processed during module load (and unload) because they are processed by the initialisation of other code. I think the following will fix this and thus allow the same driver object to be run either as a loaded module, or linked into the main kernel. 1) Add another link-set describing each link-set. At a minumum containing the name of the link-set, and functions to process the addition and removal of data areas. 2) When the module-loader is loading a module and finds a link-set (which is how it finds the module info itself), it searches the known link-set processors (from the link-set defined in (1) and any added later) and calls the relevant function to proces the entries. I think the link-set functions need initialising after the module's own initialisation - unless that is a property of the link-set? The module loader will need to remember the link sets info for module unload. That should make it easier to build some bits as modules. I have an aim that the kernel build be changed slightly: 1) build all the 'modules' *.kmod. 2) run 'config' and compile a kernel that excludes all the modules but leave the final link as an 'ld -r' generating a netbsd.o. 3) Link some or all of the *.kmod into netbsd.o 4) A final link generating a fully fixed-up netbsd. 5) Release *.kmod and netbsd.o (as well as netbsd). This would make it easy to remove a lot of 'unusual' drivers from the kernel, while still making is easy to add them into a bootable kernel for systems that need them at boot time. Thoughts/comments ? +----------------------------| Paul Goyette |--- | | I'm not sure this solves the problem. | | From my earlier attempts at modularizing the ieee80211 code, it seems | that there are two separate mechanisms for finding the start and end of | a link set: | | 1. Within a monolithically-linked kernel, we have the symbols | __{start,stop}_link_set_* but no program section table | | 2. Within a module, we have the program section table (containing a | start address and size), but no symbols | | Since the primary purpose of link_sets is to gather together some | unknown quantity of "info" from arbitrary contributors without imposing | any restrictions on quantity, I don't see how your proposal addresses | the need of figuring out the size/quantity portion of the equation. | | +--------------------------| David Laight |--- | | If you look at (say) evcnt_init() is has: | | __link_set_decl(evcnts, struct evcnt); | | __link_set_foreach(evp, evcnts) | evcnt_attach_static(*evp); | | If could also call something like: | __link_set_process(evcnts, evcnt_attach_static, evcnt_detach); | | which would save data the module loader can use to process the | program section of a lodable module. | | +------------------------| Paul Goyette |--- | | I'm obviously missing something that is, or should be, | intrinsically obvious, but... | | How would __link_set_process() know how many entries to process? | What would indicate the end of the data? | | +----------------------| David Laight |--- | | It has the program headers eg (from a random module): | | Sections: | Idx Name Size VMA LMA File off Algn | 0 .text 0000160e 00000000 00000000 00000040 2**4 | CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE | 1 .rodata.str1.1 00000105 00000000 00000000 0000164e 2**0 | CONTENTS, ALLOC, LOAD, READONLY, DATA | 2 .rodata.str1.4 000001bc 00000000 00000000 00001754 2**2 | CONTENTS, ALLOC, LOAD, READONLY, DATA | 3 .rodata 00000099 00000000 00000000 00001920 2**5 | CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA | 4 link_set_modules 00000004 00000000 00000000 000019bc 2**2 | CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA | 5 link_set_sysctl_funcs 00000004 00000000 00000000 000019c0 2**2 | CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA | 6 .data 000000a4 00000000 00000000 000019e0 2**5 | CONTENTS, ALLOC, LOAD, RELOC, DATA | 7 .ident 00000071 00000000 00000000 00001a84 2**0 | CONTENTS, READONLY | | +--------------------| Paul Goyette |--- | | Unless I've missed something in my explorations, this data | is not available for the "monolithic kernel". So a built-in | module needs the {start,end} symbols to learn where the | program sections are loaded. | | Modules that are loaded by the run-time linker _do_ have the | above info, but do not have the symbols. | | +------------------| Matt Thomas |--- | | It could if the kernel loader did what ld did. create | __{start,stop}_xxxx for each section. | | +----------------| Paul Goyette |--- | | Yeah, that would be nice. But the module linker script | doesn't do the PROVIDE() | | I suppose the kernel loader could fabricate an extra elf | symbol table for all __link_set_* program sections... | | +--------------| Matt Thomas |--- | | This isn't in the script, it's internal to ld. | | +------------| David Laight |--- | | I'm not at all sure ld uses a linker script for 'ld | -r' - so you can't add symbols that way. No reason why | the module loader can't generate them though. | | But really you need the module loader be able to | request the contents of the linkset be processed by | code in the main kernel (eg to get sysctls or event | counters added). | | +----------| Paul Goyette |--- | | Well, at least for sysctl's SYSCTL_SETUP() | stuff, you probably don't want to use the same | initialization call for modules as is used for | built-ins. The built-ins are initialized with an | explicit NULL argument passed for the sysctl_clog | argument, which makes it difficult for a module to | do its clean-up. Modular code needs to (or at least, | should?) pass a non-null module-specific clog so it | can be used during an "undo" at MODULE_CMD_UNLOAD | time. | | +--------| Jukka Ruohonen |--- | | Indeed I consider it a bug if a module uses | SYSCTL_SETUP() (and does not tear down the nodes | after unload). This applies more generally to | drivers too. | |_ | | ---| Paul Goyette |--- | | JMHO ... | |_ | |_ | |_ | |_ | |_ | |_ | | ---| David Laight |--- | | When it processes section 5 it allocates 4 bytes somewhere ... | Since the entries are all pointers (actually it ought to be | possible to use any fixes size structure) it known where the end | is. | | +--------------------| Paul Goyette |--- | | Section 5 deals with the SYSCTL_SETUP() stuff. When modules | create a sysctl(9) sub-tree, they have to explicitly call | sysctl_createv() since the initializer(s) defined by | SYSCTL_SETUP doesn't get called. | | (The code that does call the SYSCTL_SETUP() initializers | uses the __link_set_foreach() macro which depends | on the {start,end} symbols - see sysctl_init() in | src/sys/kern_sysctl.c) | | | | So, I'm still hoping to see a prototype the works for both | built-in and independantly-linked modules ... | | | :) | |_ | |_ | |_ | | ---| David Laight |--- | | Plausibly this data could be in a linkset itself! | | +------------------------| Paul Goyette |--- | | Hmm. If this data were available (at link time) in your model, it | should already be available within the existing framework. | | | Since I'm lacking a clue or two, I'll defer further comments here | until there's a prototype to work with. | |_ | | ---| David Laight |--- | | It would be an error to try to load a module which contained a | linkset without anything defined to process it. | |_ | | ---| Paul Goyette |--- | | I _do_ like part 2 of your proposal - linking the "core" kernel first, | and then re-linking with selected modules. | | +--------------------------| Matthew Mondor |--- | | I also think that this would be very nice | |_ |_