Project UDI

For those of you who are linking to this page from outside the main OZONE site, it can be seen here. OZONE is an OS I wrote myself that combines my favourite parts of VMS, Linux and WindoesNT. I am in the process of porting UDI to OZONE and vice versa.

Project UDI is a standard for writing device drivers. The goal is to have drivers that will run on any hardware and software platform that is compatible with the physical hardware. So if you have some PCI board from someone and they supply an UDI driver for it, you can just plug-and-pray!

UDI can be used as a drop in I/O system if you don't already have one. I had one, so I basically fit UDI in as another driver. The way I look at it is, that UDI is a very sophisticated piece of 'hardware' that you access via subroutine calls instead of programming I/O ports.

So, for example, I have an OZONE SCSI device driver that calls UDI to do SCSI I/O's. These things are called 'mappers' in UDI parlance. I have written the SCSI mapper (by hacking up the Linux one), but I haven't tested it at all yet. So now any SCSI controller that comes along with an UDI driver will 'instantly' work.

If you are looking for info on how to write driver's for UDI, this page is not really geared for that. This is more along the lines of all the 'behind-the-scenes' stuff that is necessary to set up the UDI environment on an operating system. I don't really have much knowledge about how to write an UDI driver!

With that in mind, I try to keep the the stuff on this page is along the lines of this analogy:

• You don't need to know how a TV works, or even what it's used for, to install one. Just plug in the cord, connect the antenna and make sure people can clearly and comfortably see the glass. Yes, you do have to supply proper power and a proper antenna.

Here's what I did / am doing to port it:

1. I got a copy of the reference library from projectudi.sourceforge.net
2. Copied the env/linux directory to env/ozone
3. Went through the modules in env/ozone one-by-one and converted the routines to call the equivalent OZONE routines. This was actually fairly painless, as OZONE had almost all the things that UDI wants. (I even at points felt pitty for poor Linux kernel programmers). The only thing I had to add to the kernel was a routine to return how much room was left on the caller's kernel stack. I didn't have to mess with the code in the 'common' directories.
4. Compiled the stuff and linked together an image called udi_env_init.oz. File udi_env_init.c is basically the 'main program' of UDI. So you may be able to start your porting effort here and port the modules that this one references, and so on until you can link with no undefined symbols. The result can be dynamically loaded or statically linked into your kernel. But at any rate, once loaded, get its start routine called so it can do its initialization stuff.
5. Compiled and linked the 'metalanguage' driver/library thing for bridge devices. This is the code that changes the public-callable udi_... routines into the per-driver xxx_... routines related to bridge devices.
   A 'metalanguage' is the formal definition of the communications between regions, called channel operations.
6. Next thing to do is get the PCI bridge driver going. This is basically a set of routines that things like SCSI and ethernet drivers call to access the PCI bus, as UDI drivers are not allowed to have direct I/O calls. I had to put in OZONE-specific calls for scanning the PCI bus and for doing I/O instructions, and also hook up the interrupt routine.
7. Next I got an 21140-based ethernet card that is somewhat supported by UDI, and compiled and loaded the network metalanguage library and the ethernet driver. I also compiled and loaded the gio metalanguage library needed by the network driver. I had to hack on the shrkudi.c driver somewhat, as it did not drive my board correctly. But this is a problem with Tulip based drivers in general, as different boards wire the chips up differently.

As of today (Feb 14, 2), the whipit test program works! It sends out packets continuously on the network. I don't have an 'unload' or anything like that so I have to reboot to stop it, though. So now I can turn the TV on.

Now I will try to get it to work with my IP stack. This is where I tune a channel in on the TV and watch.

Here are summaries of questions I have asked the Project UDI developers (so far). The answers in italics represent my understanding of the answer I got, and are in no stretch a direct quote of any one on the Project UDI development team, and are usually composites of answers.

• What routine does an OS call to pass a built scatter/gather map that describes a buffer?
  Routine _udi_buf_extern_init in mapper/linux/something.c
  It doesn't exactly do what I want, but since it is an os dependent routine anyway, I hacked it up to do what I want.
• How do I get the actual transfer length for a SCSI mapper?
  The 'ack' (request successful completion) routine is only called if the whole buffer was transferred.
  There is a status code for less-than-whole transfer, but apparently it won't tell you how much was transferred.
• How do I ask UDI what SCSI controllers it is processing?
  The SCSI mapper gets a callback for each device (controller and peripheral) that UDI sees.
  The SCSI mapper is an OS dependent routine that interfaces with UDI to provide SCSI functionality.
• Where do I get the 'channel' arg of my udi_cb_alloc calls when I allocate a scsi_io_cb?
  It gets passed as an argument to the callback routine (in the previous question).
• For the UDI semaphores, what range of values indicate a wait state, and what range indicates running?
  They are only used by the Linux port, so you may not need them. So set them up as required by your port.
• When I linked together the udi_env_init image, I was left with two undefineds, _udi_sprops_scn_for_udi and _udi_sprops_scn_end_for_udi. What are they for?
  You put the contents of env/ozone/udiprops.txt between them.
  Each line is null terminated without the newline char, and has the comments stripped out and leading/trailing/redundant spaces removed. Also, continuations lines are merged together.
• Regarding the PCI bridge driver, is it supposed to scan the PCI bus and tell UDI what devices are on the bus?
  Yes, UDI calls it repeatedly to enumerate devices it finds until it reports there are no more.
  I could not use PCI BIOS though, as it will only scan for a particular type of device, so I have to scan using brute force, ie, try all possible bus/dev/func combinations. Right now, I just scan bus 0, though.
• Then does UDI automatically load those drivers?
  Ideally, yes. But no one has implemented that on any platform so far. The SVR5 implementation does it externally, ie, its module load command is now smart enough to know to load the required libraries and initialize them when loading some driver, by looking at the appropriate udiprops.txt files.
• Is there a hard disk metalanguage?
  Not as such, but one is needed. You can use the SCSI metalanguage or invent your own.
• For a driver, what is the 'main' routine (for initialization purposes)?
  From the environment implementor's point of view, _udi_module_load. You pass it the address of the driver's udi_init_info struct, and a (length, pointer) to where you have loaded the driver's udiprops.txt file (condensed as in the udi_env_init module).
  So what I am doing now is linking in a '_start' routine with a condensed version of udiprops.txt and it calls _udi_module_load (which is part of the udi_env_init.oz image I loaded earlier).
• What's a good way to start testing my port, now that I have the ethernet driver loading? Try the whipit program. Whipit is a program which continually sends packets over the ethernet as fast as it can.

Things I've had to do to my kernel to handle UDI:

• Add a routine to return how much kernel stack space is left for the calling thread. UDI does most high-level stuff via callbacks, just in case asynchrounous completion is required. If the function completes synchronously, the callback is called before the routine returns, so the stack can get eaten up. So UDI checks to see if the stack runs into the last 4K, and if so, resets the callback nest by deferring the callback as if the completion was really asynchronous.
  This differs from the OZONE way, which is to have a synchronous completion status returned by the function, so it leaves it up to (or allows) the caller to keep the call nesting to a minimum.
  VMS's way of handling this was to always queue the completion as asynchronous so it would always reset the stack (and always incur the resultant queuing overhead). But this made the programming the simplest, because you knew the completion routine would never start until after the requesting routine had exited.
• Fix and complete dynamic kernel image loading stuff. UDI could be statically linked into the kernel, but you would have to monkey with the symbols, as there would be more than one 'udi_init_info'.
• Add a boot param to tell my native Dec tulip ethernet driver to ignore a particular interface so it will leave it alone for UDI testing.
• Fix the shared IRQ routine. Up to this point, I have only used the exclusive use routine.
• I didn't *have* to do this one, but... I changed my ethernet interface to handle ethernet packets of various formats. Before it would only handle the traditional 6-byte MAC addresses followed by 2-byte protocol followed by the data. Now, the sizes and offsets of the address and protocol fields can be specified on a device-by-device basis. The max size and offset of the data can also be specified per device. I figure I better do this now before doing the UDI network mapper so I don't have another thing to fix later.

Things that I didn't port correctly to begin with:

• The _OSDEP_EVENT_WAIT macro must clear the event flag after waiting for it. I just left it set and so the next time the daemon went to wait, it just zipped right through and went into a loop eating up the cpu time.
• The metalanguage 'libraries' really need to be loaded and called to initialize. I first thought they were just passive subroutine libraries, but they need to link themselves into UDI's list of available metalanguages so drivers will see them.
• UDI will unashamedly malloc and free from an interrupt routine. OZONE doesn't support this, so I route such mallocs and frees through the alloc daemon. I route the malloc's by failing them (causing it to do an async malloc), and the freed memory gets put on a queue for the daemon to process. If it is a performance problem, I can change OZONE, but I'd rather not have to mask out interrupts during the malloc routine for everyone just to support UDI.