You are not logged in.


#1 2005-10-28 20:26:55

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Program loading

Right now i have finished code for loading programs or drivers at any address in memory.
The thing is that I use segmentation for that, I create two new GDT selectors for each thing that
gets loaded. It works, but I really don't like it..
I know far to little about paging and program relocation etc to think about any other _really_
simple way of doing it.

I would rather use some other method since segmentation isn't very supported in other
languages then asm, and not present at all in 64-bits.. I may add some 64-bit mode to BOS
later, buying my AMD64 very soon now.. wink

So, if anyone has some ideas that i could use for loading stuff without depending on segmentation
it would be great!

/ Christoffer

Offline

 

#2 2005-10-31 01:48:34

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

from page 58 of Burgess (describing MMURTL):
"Paged flat memory is a variation on simple flat memory.  With this scheme, chunks of physical memory, called pages, are allocated and assigned to be used as a range of linear memory.  This is still a flat scheme because all the programs will use the same linear address....With this scheme, the linear address range can also be much larger than the amount of physical memory, but you can't actively allocate mor linear address space than you have physical memory.  For example, your linear address space may run from 0 to 1 gigabytes, but only 16 megabytes of this linear address range may be active
if that's all the physical memory your system has."
Here, Burgess' otherwise EXCELLENT reference shows its age a bit, since, these days, computers do not routinely contain only 16 mbytes of RAM, as they did 15 years ago, when he began working on MMURTL.
In view of the tendency to incorporate progressively LARGER amounts of memory in ordinary desktop computers, I DOUBT that PAGING is necessary for BOS.  I suggest FORGETTING about paging as antiquated, a relic of bygone days, much like segmentation--completely useless.  All segment registers should be assigned to the same location, to create the flat model, and thereafter, skillful programming will assign CODE sections, and DATA sections, keeping track with excellent documentation, rather than using obsolete segment registers.  I cannot fathom, Christoffer, why you are still using segment registers for BOS.
64 bit cpu: 
Hmm.  I haven't even got a useable 32 bit OS, yet, can't imagine starting on a 64 bit version!!!
cheerio!

Offline

 

#3 2005-10-31 14:19:54

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

Well, i guess i could figure out some executable format for BOS that can be relocatable
without using segments or paging, it's just that i think it's gonna be a bit tricky..
For example:

If BOS goes in (at program loading) and changes the programs jumps
and other addressing to add the base address (address to which the program got loaded),
then how would i  know when the program refers to "global" memory positions such as
0xB8000 or when it wants to put a 'A' in it's variable at address BASE+0x678 ??

Documents/links or info on how to best load programs without any paging or sgmentation
is welcome... I think Axel had some ideas of making a exec. format? Where are you when
I need you? wink

Offline

 

#4 2005-10-31 19:15:45

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

Maybe it is my misunderstanding.  Perhaps the key word, which I have ignored, until now, is "relocatable".  Why would BOS need to be relocatable?
Why can't BOS reside at a fixed address?
Wasn't the great need for relocatable code resolved by larger quantities of memory, that became easily acquired by virtue of their low cost?
In other words, is there some OTHER reason, now, in this era of plentiful, inexpensive memory, to create relocatable code?  Maybe I just fail to understand the scope of the problem!!!
I envision BOS loading ONE TIME, and ONLY ONCE.  Why would it be desirable to have it MOBILE, to be relocated somewhere else?
I can understand the situation, let's say, for instance, in a real time data acquisition study, outdoors, with only a small battery to supply power, that having LOTS of memory could be counterproductive, because of the need to conserve energy, due to the small size of the battery.  In that setting, where, there is perhaps, only 1 megabyte of memory (--is it possible to purchase such a small amount??), then, I can understand why someone would want to eliminate every unnecessary instruction and indeed, expel the whole operating system, to make room for the data pouring in, then, after all the data arrives, dump the data somewhere, perhaps onto a floppy, and restore the operating system on top of the same memory used to save the data.  But, I did not envision that sort of application for BOS, again, maybe I am too shortsighted.
I just do not understand why relocation is a useful property for an operating system, sitting on a desktop, with essentially unlimited power, and a huge memory available, that can be partitioned as desired.
So, my answer,  possibly completely incorrect, to your question below:
"...how would i  know when the program refers to "global" memory positions such as
0xB8000 or when it wants to put a 'A' in it's variable at address BASE+0x678 ??"
is VERY SIMPLE--
(Simple answers from simple minds!)
YOU decide where all data goes, not the operating system.
No relative programming.
ALL absolute.
No pointers, all arrays.
As I said, simple answer, maybe not to your liking.....
regards

Offline

 

#5 2005-10-31 21:51:47

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

I would like to have some drivers, maybe the command line interface and programs outside of the kernel.
They need to be loadable to diffrent addresses. I can't predict which memory areas will be free for each driver
when i may have to load many diffrent drivers/moduls, and maybe even some sort of TSR programs..

Offline

 

#6 2005-10-31 22:58:08

Nelson
Member
Registered: 2005-07-02
Posts: 35
PM

Re: Program loading

I'd maybe create a service dedicated to this as opposed to the executable format doing the relocating. (IMHO) this will be very ugly. I know the goal of BOS is to have no protection but perhaps don't use the protection offered by paging, only the benefits. smile. The Intel documents offer great insight on paging and I don't think it would be too hard to implement. (Don't know how much harder it would be in ASM, in C it was pretty easy maybe 2-3 days. Then again I'm not too seasoned in ASM)

Good luck smile

Offline

 

#7 2005-10-31 23:52:08

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

I want application to reach the whole mem, and if i use paging to let many diffrent modules
start at 0x00000000 then what if they want to do something with the realmode IDT or sth?
Maybe i could enable paging only for memory above 1mb, and map the lower mb as 1:1?
I'm trying to make it future-compatible with 64-bit and all.. Mmm.. I really don't know how to make
it simple _and_ good.. hmm
One thing is for sure, I got to get the AMD64 docs so i can look up some things.. The reason I'm
suddenly so into using some compatible method for both 32 and 64-bit is simply that i'll buy
a AMD64 around xmas and it would be nice coding some on those extra registers in the future.. tongue

/ Christoffer

Offline

 

#8 2005-11-01 00:03:56

Nelson
Member
Registered: 2005-07-02
Posts: 35
PM

Re: Program loading

I think you can just disable paging, then reenable it with the PDBR once you re-enter protected mode and everything would work dandy. Not sure about this though.

Offline

 

#9 2005-11-01 21:25:18

python
Member
Registered: 2005-11-01
Posts: 12
PM

Re: Program loading

You could design your own exacutable format like .bos or .bs something that would operate with the simplicity of .exe

Last edited by python (2005-11-01 21:34:55)

Offline

 

#10 2005-11-01 23:26:52

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

I know, just not sure how.. Especially if :

If BOS goes in (at program loading) and changes the programs jumps
and other addressing to add the base address (address to which the program got loaded),
then how would i  know when the program refers to "global" memory positions such as
0xB8000 or when it wants to put a 'A' in it's variable at address BASE+0x678 ??

I should try to study formats like ELF and such.. hmm

Offline

 

#11 2005-11-02 13:07:50

python
Member
Registered: 2005-11-01
Posts: 12
PM

Re: Program loading

Is there someway to just port the already existing .exe file to your os?

Offline

 

#12 2005-11-03 09:32:46

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

I'm not sure that EXE's is such a good idea, better to
design my own format then.

Offline

 

#13 2005-11-06 00:26:15

python
Member
Registered: 2005-11-01
Posts: 12
PM

Re: Program loading

How do you expect to write a exacutable format?

Offline

 

#14 2005-11-06 16:50:16

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

Here is the simplest format:

Code:

use32
        ORG   0x200000          ; where our program is loaded to
        jmp   start             ; jump to the start of program.
        db    'DEX1'            ; We check for this, to make shore it a valid Dex4u file.
 ;----------------------------------------------------;
 ; Start of program.                                  ;
 ;----------------------------------------------------;

start:

as you can see just after

Code:

 jmp   start             ; jump to the start of program.

is

Code:

db    'DEX1'            ; We check for this, to make shore it a valid Dex4u file.

This is checked tomake shore its a Dex4u file (as this codes for the OS Dex4u).
but most headers are full of other info like stack size etc.
This file needs to be loaded at "0x200000" for it to work properly.

Heres a example from a menuet64 file format:

Code:

use64

    org   0x0

    db    'MENUET64'              ; Header identifier
    dq    0x01                    ; Version
    dq    START                   ; Start of code
    dq    image_end               ; Size of image
    dq    0x100000                ; Memory for app
    dq    0xffff0                 ; Esp
    dq    0x00                    ; Prm
    dq    0x00                    ; Icon


START:

All file format are basically the same, but expect much more info to be layed out in a certain order, the loader uses this info to loaded the file properly.

If you do not need to relocate a file BOS would only need to do this:

Code:

        use32
        ORG   0x200000          ; where our program is loaded to
        jmp   start             ; jump to the start of program.
        db    'BOS1'            ; We check for this, to make shore it a valid Dex4u file.
 ;----------------------------------------------------;
 ; Start of program.                                  ;
 ;----------------------------------------------------;

start:
;do some code here
ret

To assembly it with fasm you could do this "c:\fasm test.asm test.bos
to run it you just need to load it to 0x200000
and than do "call   0x200000"
and it would run, but for being able to load it any where is not as simple.

PS: bubach, one way you could relocate files/modules etc, is to port fasm to your OS, find a free location in memory to load the file/module to, fill the location in the files/modules  asm file (eg: write a new  ORG   0x******** address ), assemble it and load it to that address.
This would only need to be done the first time the driver etc, is loaded.
It just keeps things simple.
The drivers asm file, could be deleted after its assembled.

Offline

 

#15 2005-11-06 21:31:28

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

menuet64 used paging which i don't want to, and having it on a absolute position isn't really want i want either.. sad
got to fix some simple relocation... i'm to picky.. wink

Offline

 

#16 2005-11-07 11:07:39

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

bubach the 64 menuet part was in answer to python, i was thinking of this as a way round the relocate problem.
" one way you could relocate files/modules etc, is to port fasm to your OS, find a free location in memory to load the file/module to, fill the location in the files/modules  asm file (eg: write a new  ORG   0x******** address ), assemble it and load it to that address.
This would only need to be done the first time the driver etc, is loaded.
It just keeps things simple.
The drivers asm file, could be deleted after its assembled."

Offline

 

#17 2005-11-07 13:25:43

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

bubach wrote:

menuet64 used paging which i don't want to, and having it on a absolute position isn't really want i want either.. sad
got to fix some simple relocation... i'm to picky.. wink

Nothing wrong with being "too picky", but, WHY do you wish to avoid having an absolute location for BOS?  Of course it is wasteful of memory, but so what?
50 years ago, relocation was not only LOGICAL, it was imperative.  Skillful use of memory was key to creating a functioning, multiuser, operating system.
Today, such complicated systems represent an anachronism, yes, some of those old ideas are VERY creative, and elegant.  But, they are also SLOW in execution, and difficult to modify.  Most importantly, the foundation upon which they rest, the paucity of memory available--with the overriding need to CONSERVE memory at all costs, is gone.
BOS need not be slow or difficult to modify.  Absolute addressing, while squandering memory by assigning fixed locations for components which may not require the vast quantity of kilobytes (!) assigned, and setting the stage for vandalism, by publically distributing the nuts and bolts of BOS, reduces the effort needed to create the OS in the first place.  While unwanted tampering will be promoted with such an approach, REMEDIATION, following the illicit intervention, will be routine, without need to devote hours and hours trying to find out HOW the villain sabotoged the operating system.  Simple, just recopy a bonafide version to the standard location, and reinitialize, wiping clean all system memory to remove whatever had been inserted by the intruder, checking the cmos concurrently to ensure that nothing had been written there since the prior authorized boot.
I am not arguing against your position, just seeking clarification.  What does BOS lose by having  specified, fixed, immobile addresses?  Yes, there are trade offs, and you may have an excellent reason for NOT using fixed addressing.   I just wonder what that reason may be?  cheers!

Offline

 

#18 2005-11-07 14:37:46

bubach
Administrator
From: Trollhättan, Sweden
Registered: 2005-02-15
Posts: 367
PM  Website

Re: Program loading

one way you could relocate files/modules etc, is to port fasm to your OS, find a free location in memory to load the file/module to, fill the location in the files/modules  asm file (eg: write a new  ORG   0x******** address ), assemble it and load it to that address.
This would only need to be done the first time the driver etc, is loaded.
It just keeps things simple.
The drivers asm file, could be deleted after its assembled.

Ah, ok.. So you mean like a absolute position for each driver/module, but specific to the
host computer.. Hmm, intresting idea..

notoka: I don't think you get my point.. i don't care that the OS, BOS is at a fixed address.
what i want is to have external drivers that can be loaded or unloaded at runtime.
so for example, if you load BOS and later wants to load a specific driver, you can,
without having to worry about which memory location to load it at or how big it is.
I could then just allocate some space anywhere in memroy for it and run it. This functionality
could also be nice for some sort of TSR programs etc, in the future...

So, does this clarify it?

/ Christoffer

Offline

 

#19 2005-11-09 16:50:11

notoka
Member
Registered: 2005-09-04
Posts: 12
PM

Re: Program loading

bubach wrote:

...what i want is to have external drivers that can be loaded or unloaded at runtime. so for example, if you load BOS and later wants to load a specific driver, you can, without having to worry about which memory location to load it at or how big it is.
I could then just allocate some space anywhere in memroy for it and run it. ...

Hmm.  perhaps I am still confused.  Don't wish to waste your time.  Please ignore the following, as you prefer....
Suppose a new "driver" is added to BOS, something to permit widget xyz to operate, maybe a video camera connected to USB-2, or whatever.
As I understand your explanation above, you will still be assigning a specific memory location for the new videocameradriver code.  The issue is whether or not this particular chunk of code could, or should, be able to be relocated to some new location, (NOT THE ORIGINAL LOCALE SPECIFIED BY YOU,) by BOS.  I think it is unnecessarily complicated to add the ability of the operating system to relocate components of itself.  What does one gain upon adding this capability?  What does one lose,  if the videocameradriver code is sitting in a fixed location, NOT ABLE TO BE REPOSITIONED by the operating system?
cheerio

Offline

 

#20 2005-11-09 19:09:59

Dex4u
Active member
Registered: 2005-02-19
Posts: 132
PM

Re: Program loading

notoka, i think bubach needs to be able to relocate a driver because if not, he will need a space for every type of driver and even then how much space should he leave for each driver ?.

I think a simple way would be to have drivers as asm code, with the org left blank, then free space can be found in memory, the address is than add to the org and assembled with fasm.
He will need to port fasm to BOS, but he is doing that any way.
As fasm is a fast assembler it will take no more than a second or two to do this.

Offline

 

 

Board footer

Powered by PunBB
BOS homepage © Copyright 2005 Christoffer Bubach
Strict XHTML and valid CSS.