You are not logged in.


#21 2005-11-10 16:33:17

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

Dex4u wrote:

notoka, i think bubach needs to be able to relocate a driver because if not, he will need a space for every type of driver and even then how much space should he leave for each driver ?.
...

As much space as he wants.
Space is PLENTIFUL.
It is no longer 1958.
We do not need to CONSERVE memory.
ASSIGN 10x more space than one can imagine might be required, and then increase the whole space ten fold.
No, don't bother.  Just give the entire operating system 16 megabytes.
What's the point of worrying about a few tens of kilobytes?
Cheers.

Offline

 

#22 2005-11-11 10:27:25

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

Thats not the point.
Programs are compiled/assembled to start at a sertain address in memory
and without some kind of trick like paging or so, you can't load more then
one driver, becasue you can't load everything on the same address.
So the only time where i can specify where programs should be is when i compile
them, not when loading them.

So somehow BOS must go in and change the program (after loading it to RAM)
so that it knows where in memory it is.
For example:
I code a driver for BOS and all drivers have "org 100h" which means that they expect to
be loaded at address 0x100.
BOS allocates memory at maybe 1mb for the program and loads it there.
Now if i start running the program all jumps and variable read/writes would point to 0x100
instead of where i actually loaded it, which must be fixed.
Most OS's use paging which can trick all programs and drivers to think that they all resists
at the same memory location, when in reality they are somewhere else.

Offline

 

#23 2005-11-11 12:53:33

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

bubach wrote:

Thats not the point.
Programs are compiled/assembled to start at a sertain address in memory
and without some kind of trick like paging or so, you can't load more then
one driver, becasue you can't load everything on the same address.
So the only time where i can specify where programs should be is when i compile
them, not when loading them.
So somehow BOS must go in and change the program (after loading it to RAM)
so that it knows where in memory it is.
...

Hmm...
I can see that my question is not well formulated.
There is some code for keyboard, some for video display, some for accessing the ide controller, etc.
Surely not all code begins at "org 100h".  The entire BOS can commence at 100h, or any other address one chooses, in my opinion.  I don't understand why BOS must relocate various components of itself.  Having both floppy drivers and hard disk drivers (i.e. the ability to retrieve data from either type of storage media) does not imply, at least to me, that the two sections of code associated with retrieval from those two specific media MUST, a priori, both be located at "org 100h"--thence to be reassigned to some UNKNOWN location, by BOS.  Why make BOS so complicated?
With regard to the question, yesterday, re: length of the two modules, or, more accurately, assigning a length, without knowing how much space to assign for floppy drive controller, and how much to assign for hard disk controller, my answer is perhaps TOO childish:  Assign a LOT of space.  Much more than could ever be imagined.  With fixed addresses for these two components, one loses MEMORY, i.e. WASTES memory, because lots of the space assigned to the two drivers is just empty.  So, yes, there is an advantage to having BOS assign the location, instead of the architect of BOS, BUT, that "advantage" disappears if one realizes that there is ample memory, even with so much wasted space.  The real advantage we should seek, in my opinion, is saving DEVELOPMENT TIME, rather than saving memory.  I believe, perhaps INCORRECTLY, that it is much simpler to debug anything, if it resides at an address that is both FIXED, AND KNOWN, and not one assigned on the fly, by BOS.  The cost of FIXED addresses, is memory wastage--wasted because more space is assigned, than is actually used.  The benefit of having BOS establish an unknown location for various components of BOS, to save memory, is ANACHRONISTIC.  We need not save memory.  We need DEVELOPMENTAL simplicity, and developmental efficiency, and BOS sufficiently facile that others can assist in its elaboration.  Did not this thread commence with a question relating to finding a simple solution to a complex problem???  Why not solve the problem by eliminating it?  What is the NEED for relocatabiliy?  I appreciate that it is TRADITIONAL, to develop operating systems that way, but is it NECESSARY?  Isn't the only price one pays for FIXED LOCATIONS, the cost of wasted memory, which in 2005, we can readily afford?  Apart from SAVING MEMORY, is there some other advantage associated with assigning BOS the task of locating components of itself?   For example, is BOS FASTER in execution, because of this "relocation capable" feature?   If BOS type one has fixed, architect assigned addresses, and BOS type two has relocatable addresses (assigned by BOS), which version will be easier to modify in the future?  hope you have a good day!

Offline

 

#24 2005-11-12 04:13:32

Nelson
Member
Registered: 2005-07-02
Posts: 35
PM

Re: Program loading

imho too much trouble as opposed to implementing paging. Just implement it with no protection (All programs running in same context) No one will notice!

Offline

 

#25 2005-11-22 20:17:56

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

Hi,

I only read it half by half, so if someone said the exact message, don't bother.

RE relocatable drivers/programs/all:

IMO this makes it more complex when programming in Assembly. Now in C or any other HIL it would not make any difference, because the compiler would take the job on itself.

RE programs and relocation:

I think, to remove the complex part you can use paging. Why is it so hard to implement? Don't forget, you only have [b]one[/b] program running. Nothing more, well I at least don't expect to have more now does it?

RE drivers:

I reckon you should have some sort of relocation.

1. Use headers and with the offset header, code your code
2. Don't use headers at all, and get the EIP yourself.


If you don't want to have these, program a programming language with a proper compiler and you're all set smile.

HTH a bit,

DennisCGc.

@notoka: the reason why drivers should be relocatable is that you really don't know where the driver is going to be loaded, at least the programmer does NOT. The OS CAN NOT determine the memory layout, because every driver has a difference size. (do you know what I mean?)

Offline

 

#26 2005-11-23 13:45:03

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

Thanks Dennis,
   No, I guess I am just kind of ignorant.  What is a "driver"?  Isn't it just a collection of commands aimed at controlling the operation of some device?  Why must this group of commands be relocatable?  Why can't this component of BOS be assigned a specific address, some place sufficiently far away from the other operations, that the collection of commands could not overlap in memory with some other commands?  I appreciate that doing this would involve WASTING a huge amount of memory, but that was not my concern.  IS there SOME OTHER reason, apart from NOT WANTING to waste memory, that necessitates relocation capability?  You suggested that the programmer could not assign a specific physical address, because of uncertainty about the size of both the driver, and the other components of BOS, in order to situate the code for the driver in a locale sufficiently remote that there is no danger of overlap with either other parts of BOS, or the data manipulated by the OS or one of its applications.  I just cannot visualize that problem.  In a test scenario, for example, where there were "only" 256 megabytes of memory on the motherboard, one could place all the rest of BOS at the first megabyte, and the new driver at a position 128 megabytes away, and still leave at least 127 megabytes, AFTER the last command in the driver, for data.  Then, it would be straightforward to measure the length of the new code for this new driver, and reassign it, manually, to a new location, closer to the rest of BOS, thereby freeing up another 127 megabytes, for more data.
So, is it just a question of TEDIUM, i.e. boring to engage in such manoeuvres, much more stimulating to build relocation capability, than to plod along like a farmer measuring his field, assigning oats, barley and wheat to various quadrants?  Or, alternatively, is there some OTHER reason, not related to memory, not related to boring tediusness, some other factor which necessitates implementation of a relocation capability?

Offline

 

#27 2005-11-24 22:02:32

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

Hi,

No, I guess I am just kind of ignorant.  What is a "driver"?  Isn't it just a collection of commands aimed at controlling the operation of some device?

Yes, absolutely correct!

....

Okay, I try to explain it all clear... it's just me who can't explain (so don't blame yourself)...

What if you have 2 drivers? Let's say a USB driver, compiled at 0x10000, and it takes about 0x1000 bytes. What if the user wants to load a second driver (let's say, it's a sound card driver)? It should be okay, the driver is compiled at 0x11000 and it takes about 0x1000 bytes. No problems here. But wait! The user wants to load a third driver. Unfortunately, the second and third drivers were developed by other people and they thought address 0x11000 was free. Now we have a problem. (note, this is just an example, I can give you other examples.) Because we seriously can't load both drivers at the same address. It would overwrite eachother.

Now you could say we could just use more memory. But then again, every single driver has to have a unique address assigned. And what if somehow the USB driver grows in size? Then you would really have to change each driver. IMO this is too much work.. and really not useful (compare it to Windows, Windows has really more than 1000 drivers available)

Then there's relocation. Relocation is the technique which makes a code almost indepent of the position where it runs. Of course it's more difficult to implement. But it's more efficient.

I think your mixing up things a bit. Most code IS position DEPENDANT. Really.. well, at least it's a lot less dependant when using paging, which Bubach wants to avoid at all cost. You really should realise that jumps to other labels most of the time are not position dependant, but accessing offsets IS position dependant. Ever tried assembly? If not, maybe you should at least try to understand it a bit. You really don't have to go into the protected mode commands, etc. Just doing the 'easy' stuff like mov, add, sub, jmp, and a few others as well. You're going to see that code is a lot more position dependant than you thought it would be smile

No, I'm not at the stage of designing my loadable/unloadable driver support. And I don't want to have it, at least not now. I'm rather busy atm with other things. (EG. Getting my process manager up, etc.)

Hope I made myself clear.

DennisCGc.

Offline

 

#28 2005-11-24 23:27:49

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

I still think using fasm is the simpleist way round this, all you need do is have drivers as asm code, find some free memory space, pach the org to match the memory address, assembly it and load, as fasm is so fast, it will take about 1 second to do the above.
You should only need to do this once, for each driver. for each pc.

You will need to port fasm, but you will be doing that anyway wink.

Offline

 

#29 2005-11-25 14:53:24

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

mm, compiling at runtime? it's an option...  but IMO it would seriously downgrade the performance of driver loading...

PS. It's common to ignore some posts? </sarcasticandjustbeinghopefullyabitfunny>

Offline

 

#30 2005-11-25 15:34:55

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

What about giving each driver a LDT, So the base is the load address, so instead of a 0 base, it would be the load address, and the driver bin file are set to org 0.

Offline

 

#31 2005-11-25 17:51:50

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

DennisCGc wrote:

....
The user wants to load a third driver. Unfortunately, the second and third drivers were developed by other people and they thought address 0x11000 was free. Now we have a problem. (note, this is just an example, I can give you other examples.) Because we seriously can't load both drivers at the same address. It would overwrite eachother.
....DennisCGc.

Thanks Dennis, yes, I agree with you, that multiple drivers cannot access the same address.  Further, I share your concern that different individuals could unknowingly assign THE SAME address, by mistake.
I solve that problem by DOCUMENTATION.  If there is a suitable header at the start of BOS, identifying the various drivers, by name, and starting address, plus approximate length, then, when you come along, wanting to introduce your own custom USB driver, it should be simple enough to send a message to whoever is managing the system, notifying the world at large that you intend to place your new USB driver at 0x22200.  "Speak now, or forever hold your peace".  Well, not really.  If it turns out that joe or fred or sam or sally has ALSO written a new driver, and used precisely the same address, or one nearby, that potentially could interfere with operation of your driver, then you would simply "relocate" your code to, for example, 0x58000.  How difficult was that?  Took what, 30 seconds to REASSEMBLE (not recompile, for there is no compiler, only an assembler--this is not C or fortran or Pascal.), and RELINK, (unless the linker is integral with the assembler--but n.b. it is still incorrect to call that process "compilation".)  This operation does not depend upon any particular assembler, any of them will work fine.

Offline

 

#32 2005-11-25 19:31:30

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

Dex4u wrote:

What about giving each driver a LDT, So the base is the load address, so instead of a 0 base, it would be the load address, and the driver bin file are set to org 0.

Now that's an idea which should not be forgotten smile Or you just could give create a new LDT and then you could 'put' the drivers there..
Or your other interesting idea could be sufficient.. (the recompile idea)... although I doubt it's the 'perfect' solution...

Really... just imagine this...

Your OS is popular enough that the big companies will write drivers for your OS... but there's a major problem... do you really think those other companies will open-source their drivers? I really don't think so.. Although it's well thought, you NEVER gonna get the company support you actually want...

Offline

 

#33 2005-11-25 19:45:46

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

notoka wrote:

I solve that problem by DOCUMENTATION.

Are you kidding me or what? Come on, I already said Windows f.e. has about more than 1000 drivers. How would you like to have that implemented? It's IMO just a waste of time. Dex4u's solution is one of the better alternatives of relocation.. You simply CAN NOT force others to 'register' their drivers' address... (just think about it!)

notoka wrote:

If there is a suitable header at the start of BOS, identifying the various drivers, by name, and starting address, plus approximate length, then, when you come along, wanting to introduce your own custom USB driver, it should be simple enough to send a message to whoever is managing the system, notifying the world at large that you intend to place your new USB driver at 0x22200.  "Speak now, or forever hold your peace".  Well, not really.  If it turns out that joe or fred or sam or sally has ALSO written a new driver, and used precisely the same address, or one nearby, that potentially could interfere with operation of your driver, then you would simply "relocate" your code to, for example, 0x58000.

Just what the hell was MY point before? Exactly, relocation...

Ever considered that a driver can grow?! (You design things as they were 20 years ago, give everything a limited memory region and don't think about the future... reminds me of the following thing: "640k ought to be enough for everyone" - Bill G.)

notoka wrote:

How difficult was that?  Took what, 30 seconds to REASSEMBLE (not recompile, for there is no compiler, only an assembler--this is not C or fortran or Pascal.), and RELINK, (unless the linker is integral with the assembler--but n.b. it is still incorrect to call that process "compilation".)  This operation does not depend upon any particular assembler, any of them will work fine.

First, why actually BOTHER assembling and just RELINKING the whole again? You AND save time for the drivers to load and you save yourself time. Like I said before (read my previous posts), it takes [b]more[/b] time to load drivers.

This is OS design, why do you think not every programmer writes an OS? Exactly, it takes time to design a nice driver-interface (or just any interface). There are some 'programmers' who actually think it's EASY to create an OS (aka, they are litterally copying it from a tutorial and they are actually having questions about why it doesn't compile... )

I really can't believe you want to have EVERY driver a unique address... which IMO is a Really-Bad-Thing(tm).

I'm deeply sorry if I sound a bit harsh, I really don't mean the things like that...

DennisCGc.

Offline

 

#34 2005-11-25 22:42:05

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

Another  way is a simple list in the header, of the drives bin file, with the offset  from the beginning of the file, to all the address that need paching, also in the header could be the number of address that need paching.
you could use code like this to pach them, once the driver is loaded in memory

Code:

mov esi,[LoadAddress] ; where the driver is loaded
add esi,8 ; The offset to number of address that need paching
lodsw
mov cx,ax ; cx = number to pach
mov [PachListAddress],esi ;esi now is pointing to the first offset in the list
@@:
mov esi,[PachListAddress] 
lodsd ;move one of the address that need paching in to eax
mov esi,[LoadAddress] 
add esi,eax
mov edi,esi
lodsd 
add eax,[LoadAddress] ; add the base to pach the address
stosd ; than then put it back
add [PachListAddress],4 ; add 4 for the next address in the list
loop  @b ; loop for the number that need paching.

With out comments

Code:

mov esi,[LoadAddress]
add esi,8 
lodsw
mov cx,ax 
mov [PachListAddress],esi 
@@:
mov esi,[PachListAddress] 
lodsd 
mov esi,[LoadAddress] 
add esi,eax
mov edi,esi
lodsd 
add eax,[LoadAddress] 
stosd 
add [PachListAddress],4 
loop  @b

PS: Not tested and could be more optimized.

Offline

 

#35 2005-11-26 00:19:29

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

lol, now you wrote (manually) a linker generated asm-out file.... (IIRC, that's the way a linker works)

Offline

 

#36 2005-11-26 00:31:51

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

Its also the way Dos loads exe file wink.

Offline

 

#37 2005-11-26 01:12:21

DennisCGc
Member
From: Zoetermeer, The Netherlands
Registered: 2005-03-01
Posts: 49
PM  Website

Re: Program loading

*slow-chat: .exe's are linked...*

By the way > DOS uses segments... (it uses the fact that org can be 0 while loading it to 1000h:f000h and the program will see 1f00h:0000h, that's just using a technique of real mode)

So in DOS the origin can be either way specified, DOS will find a matching address...

We're getting quite Off-topic, aren't we?

Offline

 

#38 2005-11-26 11:16:04

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

DennisCGc wrote:

We're getting quite Off-topic, aren't we?

We are still on the topic of differant ways, to relocate driver files.

Offline

 

#39 2005-11-26 17:32:22

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

Ways to position different components of the operating system, EFFICIENTLY, from a cpu point of view:  (i.e. to me, the question is not "HOW BEST TO RELOCATE DRIVERS", but rather, whether or not FIXED VERSUS RELATIVE ADDRESS LOCATION yields a superior result.)
The question I am asking, is whether it is faster or slower, from an execution point of view, not from a development time point of view, to have ABSOLUTE, FIXED locations for each of the twenty-2000 various "drivers" that comprise the operating system.  The majority opinion appears to answer NO.  NO, an operating system will operate FASTER, if it has been designed to permit relocation by the OS, with each module positioned (also more efficiently, from a memory organization and utilization point of view) IN AN UNKNOWN place, or at least unknown to the humans.  The hundreds (or thousands) of various drivers would be situated somewhere, but it is NEITHER necessary, nor advantageous to have those locations identified, and fixed by humans.
Shouldn't such an hypothesis have been tested?  Isn't there some data available with a very small, even microscopic, operating system that accomplishes little more than boot from memory with display hello on the screen, measuring the time needed to boot up, and display that one line message with the FIXED versus RELOCATABLE positions for the half dozen elements of such a small operating system?   What elements are needed for that microscopic operating system?  timer access, ide controller, keyboard, video controller, interrupt controller, well, that's at least five "drivers" plus the kernel.....Preemptive versus cooperative?  Don't know.  Access all hardware from protected mode, versus switching back and forth to real mode?  Don't know.  Ring zero entirely would seem the logical best, reducing time needed to switch modes.  (We would not want to test this hypothesis with a "noisy" test model, i.e. if the time needed to switch back and forth between real and protected mode is MUCH greater, than the tiny time difference measured with relocatable versus fixed location drivers, then one might need to access all motherboard devices independently from BIOS, however, to remain continuously in protected mode, rather than incur the time penalty for going back and forth to real mode.  This would mean writing directly to the video controller, for example, and would then pose a challenge if the video controller innards were undocumented.)
     I don't know the answer.  On the other hand, perhaps, notwithstanding some test data confirming the validity of my notion that fixed addresses for drivers correlates to faster execution, the resultant operating system may be so ungainly, and so unmanageable, as Dennis has suggested, that even if it enjoyed (and I don't know any data to support the argument) a slightly speedier execution, compared with the same one line text display operating system designed with relocatable capability for those five "drivers" (i.e. by definition, MUST have additional overhead, therefore slightly, but measureably slower execution), it would be essentially USELESS to continue with this design further, as one could not imagine adding future, further capability from such a fixed address operating system design.  I have not read Tanenbaum's book. 
http://vig.prenhall.com/catalog/academi … 88,00.html
Off to the library in search of an answer, if there is one, to this question.   Wasn't there supposed to be a link to Tanenbaum's book on this forum?  A few months ago, when I joined, there was some confusion about that.... Hope my question does not appear similar to the question of the quantity of fairies capable of dancing on the head of a pin.  Maybe there is no answer to my question, in Tanenbaum, or anywhere else, both because the question itself is so banal, and also, because the answer, once procured, would not change the intent to continue on with RELOCATABLE drivers, ANYWAY.  I am eager to read more about Tanenbaum's new Minix III, maybe there is something relevant to this question there:
http://www.cs.vu.nl/~ast/
I think it was Linus Torvalds who modified Minix to permit it to operate on a 386, 15 years ago?
http://www.linux10.org/history/

Offline

 

#40 2005-11-26 18:11:26

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

DennisCGc wrote:

....
I really can't believe you want to have EVERY driver a unique address... which IMO is a Really-Bad-Thing(tm).....

Maybe it is bad thinking on my part.  I suppose that a REALLY HUGE driver, one that was poorly designed and badly implemented, MIGHT need 1k.  That's 1024 different, consecutive 32 bit memory locations, to contain the INSTRUCTIONS required to access various components on the motherboard, for example, USB controller, IDE controller, timer, keyboard, etc.  In common parlance, that corresponds to 4k bytes.
And, let's acknowledge that there is considerable merit to Dennis' reminder of a DRIVER GROWING WITH TIME.
So let's double that:
8k bytes.
Now let's keep the arithmetic simple, and call that 10k bytes.
Now, let's insert another 10k bytes, separating EACH driver, to have some space there, to ensure that there is no conflict, no overlap: 
20k bytes.
Now, let's multiply that by 1000 drivers
20 mega bytes.
Wow, the main guts of the operating system occupy only 20 megabytes.  There isn't a computer sold anywhere in the world, today, in 2005, with less than 128 megabytes of memory.  By this time next year, that figure will be 256 megabytes of mainboard RAM.  So, what I am proposing is the DEGRADATION of the system memory, by a factor of 10%, i.e. an operating system that employs 24 megabytes just for itself--though of course, most of that memory is unused!  Point is, the memory required by this operating system WOULD NOT BE AVAILABLE to the user.  So, in that sense, system memory will have been degraded.
But what will have been GAINED, as a result of this profligate use of memory?  For one thing, we will know THE PRECISE location of every component of the operating system.  This will make it very easy, (or, at least EASIER) to CHANGE individual components, as they come to be regarded as inefficient or slow or clumsy.
Secondly, I believe, maybe incorrectly, that such a design will be FASTER in execution than an operating system with relocation capability.  I have no evidence to support this notion.
As far as keeping track of all these (1000) drivers, the task is not so difficult, in my opinion, because the locations ARE ALWAYS EXACTLY 20K bytes apart.
Of course, if there were a driver LARGER than 10kbytes, this notion would be useless!!  It would be easy to refute this idea:  anyone with an example of a useful, VERY LARGE driver, could repudiate such a scheme in an instant.
I don't believe that a useful operating system, functioning entirely at ring zero, i.e. without protection, without passwords, without logins (both required in the era when a thousand people were sharing the same ONE cpu), requires ONE THOUSAND DIFFERENT DRIVERS.  I cannot count more than 100 useful hardware appliances, including video cameras and flash memory, and firewire devices, etc, that such an operating system need support.
MOST OF OUR THINKING is based on suppositions that were accurate half a century ago, but not today. cheers.

Offline

 

 

Board footer

Powered by PunBB
BOS homepage © Copyright 2005 Christoffer Bubach
Strict XHTML and valid CSS.