Date: Tue, 20 Feb 90 15:20:52 PST From: owen@Sun.COM (Owen Densmore) To: don@cs.UMD.EDU Subject: Re: NeFS > They changed the meaning of some of the standard PostScript operators, > like read and write, which I don't think was a good idea, for several > reasons... They should have used different names, or at least made .. Agreed. And I DO see reasons for the old operators. They could be optimized as a local cache for NeFS to use in its own calcs. > Basically, NeFS is a *particular* application of an abstraction of > NeWS. The abstract idea is that of having a server with a dynamically > extensible interpreter as an interface to whatever library or resource > you want to make available over the network (let's call it a generic > Ne* server). Very true. This has been particularly difficult for me to get across to others here at Sun. I recently wrote it up for Steve MacKay and include it at the end of the message. > It's not clear to me if NeFS supports multiple light weight PostScript > processes like NeWS. I asked Brent about this, and he agreed that it's an issue. Brent has been talking to a guy here who's interested in re-writing the NeWS interpreter to be much easier to program and debug. I'd love to see them come up with a NeWS Core that could be used as a generic NetWare core. I think you should send your comments off to nfs3 & see what happens! I agree with most of your points. Owen Here's the memo I consed up for MacKay: ====================================================================== Window System? ..NeWS ain' no stinkin' Window System! -or- Swiss Army NeWS: A Programmable Network Facility ====================================================================== Introduction ---------------------------------------------------------------------- NeWS is difficult to understand simply because it is *not* just a window system. It is a "Swiss Army Knife" containing several components, some of which contribute to its use as a window system, others which provide the networking facilities for implementing the client-server model, all embedded in a programmable substrate allowing extremely flexible and creative combination of these elements. During the initial implementation phase of the Macintosh LaserWriter software, I temporarily transfered from Apple to Adobe working closely with John Warnock and other Adobe engineers. At lunch one day, I asked: "John, what do you plan to do after LaserWriter?" His answer was interesting: PostScript is a linguistic "mother board", which has "slots" for several "cards". The first card we (Adobe) built was a graphics card. We're considering other cards. In particular, we've thought about other network services, such as a file server card. He went on to say how a programmable network was really his goal, and that the printing work was just the first component. His mentioning using PostScript for a file server is particularly interesting: Sun's next version of NFS is going to use PostScript with file extentions as the client-server protocol! This paper explores NeWS in this light: as a Programmable Network Facility, a major part of Sun's future networking strategy. NeWS Networking Components ---------------------------------------------------------------------- NeWS has realized John's notion of a programmable network facility. It has populated the initial PostScript mother board with several cards vital to netwoking: -Device Independent Graphics Primitives: The key importance of the PostScript graphics model for networking is that it is device independent, able to represent with one file the same drawing on *all* the network hosts. The Folio fonts contribute to this: they can be rendered at a given point size on all screens, regardless of resolution. -Host Independent Interpreter: PostScript is an interpreted language that runs on all Sun workstations without recompilation. The significance of this for networking is that small code fragments can run on any host without any knowledge of the host's architecture. Thus one host sending a code fragment to another host need not be concerned whether that host is a Sun-3, Sun-4, or even Macintosh! This extends to client programming languages as well: all client languages have the same interface to the host. As an illustration of this point, both C and Lisp clients use the TNT toolkit with no special interfaces for each language. -Network Socket Primitives: NeWS has added primitives for listening for connection requests, for socket files, for host name and internet number management, and for reading and writing on the network connection. In fact, the NeWS "server" code is not written in C, it is written in PostScript. The initial version, and the one I run on my own machine today, is 10 lines of code! -Light Weight Processes (LWP): A network host has many independent activities taking place at one time. Each of these uses a separate LWP. These activities need not be window oriented, and in fact most are not. Consider WUE alarms, for example. A LWP could be created on each WUE NetStation whose only task was to listen for activities on other WUE NetStations and to cause a WUE Alarm to occur when an interesting event occurs. -Garbage Collection: When data is allocated within NeWS, it is returned to the free memory pool when no process references it. This is needed in a multiprocessing network host because separate LWPs can reference the same shared data. (No process can safely free data itself, because other processes might reference it.) Garbage collection manages this problem for the processes, and thus is an important Network facility. -Programmable Events: NeWS has a flexible, very easy to use event management scheme. Processes fill out templates of events they are interested in. When these are matched by events occuring within the system, they are delivered to the interested process. Events are *far* more useful than simple keyboard and mouse events. They can be artificially manufactured for any sort of inter-LWP communication. For the previous WUE Alarm, for example, the WUE "Listening" process would communicate to WUE-Friendly applications by filling out a WUEAlarmEvent and sending it. The interested applications would have filled out a template which would "catch" the event. -Programmable Error Handling: NeWS uses the standard PostScript exception handling, extended to work within the context of LWPs. Thus an error in one LWP is isolated to that process only. Network programs can "wrap" incomming code inside of the special "stopped" error handler which will catch errors occuring inside the code fragment. This is vital to safe network programming. What IS NeWS Anyway! ---------------------------------------------------------------------- "Well, I'm glad NeWS has all these useful parts, but just what IS NeWS anyway; how do I get at and use all this stuff." Well you may ask! It does seem confusing. Formally, NeWS is simply a Unix command, just like "ls" and "cd". Its syntax is: xnews [PostScript-code] i.e. it is a command that executes PostScript code fragment. If the xnews command is given without an explicit argument, it defaults to executing "(NeWS/init.ps) run", which simply looks for the file "init.ps" in the "$OPENWINHOME/etc/NeWS" directory. It is *this* file that uses the NeWS components to setup the X11/NeWS window system. As a silly example of handing in your own commands to NeWS, we can do this: bigmac% xnews "1 1 add =" 2 This simply tells xnews to add 1 & 1 and print the results. Oddly enough, I have written a somewhat more complex script to fill out my Traval Advance forms, adding up all the rows and columns and printing out a version of the form! Now that we've gotten though the fundamentals, the rest of the paper will look at individual examples of using NeWS in novel ways. Although we present programs to illustrate how things are done, the reader may skip over them, the accompaning text will explain what the program does. The Ten-Line Server ---------------------------------------------------------------------- As mentioned above, "xnews" is simply a Unix command that executes a PostScript program, defaulting to "(NeWS/init.ps) run". Init.ps performs several initialization tasks and executes the commands in various other files. When its done with all of that, it defines and executes a small procedure called "server". This program uses the NeWS networking facilities to build a LWP which listens for connection requests from "client" programs. Whenever a request is heard, the LWP awakens and creates and initializes a new LWP just for the client's usage. Here's the ten line version of the server. This actually works (I've run it for quite some time!), and is explained in Chapter 10, Networking NeWS, of the Unix Networking book published last year. /server { % - => -; create the NeWS client-server listner process { clear newprocessgroup % -; Clean stack, init process group (%socketl2000) (r) file % soc; Open socket for listening { dup acceptconnection % soc f;Loop forever creating clients { exch pop 200 dict begin % f; Create userdict initmatrix newprocessgroup % f; Init graphics & process cvx exec % -; Execute the connection! currentprocess killprocessgroup%Cleanup client & children } fork pop pop % soc; Clear process & file } loop % soc; Loop forever creating clients } fork pop % -; Fork server & toss process } def The significance of this is that we can just as easily build other network services with NeWS, using code much the same as the above. NeWSPrint, for example, uses a similar PostScript program to provide network printing facilities. NeWSPrint does not require a special, modified version of xnews; it simply uses the standard OpenWindows product. ps2bits: A PostScript to Rasterfile Program ---------------------------------------------------------------------- A second use of running xnews as a Unix filter is illustrated by the ps2bits probram. This Bourne shell script reads a PostScript file on its "standard in", and emits the resulting Sun raster file on its "standard out". The shell script does this by having xnews: -Read in a few standard PostScript initialization files, -Initialize the PostScript environment (& fix a Folio bug!) -Build an 8.5 by 11, color, 72 dpi resolution "page" canvas -Execute the PostScript stdin file to draw that page -Write the resulting image to stdout as a raster file Here is the program: #! /bin/sh xnews " (NeWS/pack.ps) (r) file cvx exec (NeWS/basics.ps) (r) file cvx exec (NeWS/redbook.ps) (r) file cvx exec 500 dict begin false setautobind () stringwidth pop pop 612 792 8 [1 0 0 -1 0 792] null buildimage setcanvas (%stdin) run clippath (%stdout) (w) file writecanvas " Note: The line with the "buildimage" command may be modified to convert the program to black & white, or to change the resolution to 300 dpi, or to change the page size. See Appendix A for a more general script. Once the image is made, it can easily be looked at in SunView, XView, or NeWS using any of their raster file viewers. Here's a trivial viewer, "bitwin", using the TNT toolkit: #! /bin/sh psh < /tmp/bits bigmac% bitwin /tmp/bits & The first command converts the tiger drawing into a raster file. The second creates a window for viewing the raster file. Although the complexity has been fairly minor in the previous examples, we've illustrated some powerful capabilities: -We can easily use NeWS to provide a simple Unix facility for converting PostScript drawings into Sun raster files. The PostScript may come from any of the many widely available sources, even the Macintosh. -These raster files can be used by any toolkit, including SunView. They can even be used by other raster devices such as printers. -Using the networking primitives used in the 10-line server, we could easily build a similar server for providing raster files to anyone on the network. Using these NeWS facilities in similar creative ways is exactly what NeWSPrint does. The documentation folks use a similar "filter" to strip out all the general header information in Mac PostScript files to gain an average 8-1 compression, vastly reducing their disk storage requirements. A homework assignment in our Usenix NeWS tutorial is to create a generalized Image Server for the network using these facilities. WUE Alarms with Programmable Events ---------------------------------------------------------------------- WUE has the notion of "alarms" which can be used to alert applications of changes in status of network objects. Applications can use this for anything from "hot links" to file status alarms. One of the WUE examples is on file status: I want to know when the monthy report reaches 90% completion. Hot links are similar: I want this graph to change whenever the associated spreadsheet changes. Here is one way to implement WUE file alarms using NeWS programmable events: -WUE provides a Property Sheet which includes a file completion field holding a percentage quantity. -When the Property Sheet is completed, it creates and sends a WUEStatusEvent for the given file. The network name of the object is stored in the event's Action field. The event includes auxillary information about the file in the ClientData of the event. -Applications following WUE object status changes use the NeWS "expressinterest" primitive to catch all WUEStatusEvent's for the desired file. It then looks at the ClientData to determine whether or not further action should be taken. The WUE Property Sheet would build and send the event this way: createevent dup begin /Name /WUEStatusEvent def /Action (/home/project/status/january) def /ClientData 1 dict dup begin /Completion .70 def end def end sendevent The application builds an interest for its TNT event manager to handle. It does so by building a matching event template for WUEStatusEvent and the /home/project/status/january filename. It also includes a "callback" procedure which decides what to do with the completion information. In this case, we want the alarm to occur when the file is more than 90% complete: createevent dup begin /Name 1 dict dup begin /WUEStatusEvent { % event => -; check if complete yet /ClientData get /Completion get .90 ge { AlertApplication } if } def end def /Action (/home/project/status/january) def end MyEventMgr expressinterest This results in the AlertApplication procedure being called when the /home/project/status/january report is within 10% of completion. Puny WOF (Workgroup Object Facility) ---------------------------------------------------------------------- The above alerting works only for applications running on the same workstation. This is because the event that is sent is only visible to interests expressed on the same workstation. We can easily extend our model to include sending the event to many network workstations by using the same techniques as the 10-line server: -WUE friendly workstations start a WUE event listening service when they start up NeWS. This uses exactly the same techniques as the 10-line server, but using a different socket file. They also regester themselves with a centralized list of WUE friendly workstations. -The WUE event listening service starts a second process which expresses interest in all local WUE events. When any WUE event is caught, it is "forwarded" to each of the workstations listed in the central list. It does this by opening a socket file to each workstation, and sending a small program down the socket which sends the WUE event on the remote machine. -The Property Sheet sends the WUEStatusEvent exactly as before. Local applications running on the same machine operate exactly as before. In addition, however, the local WUE event forwarder causes the same event to be sent on each of the regestered workstations. Applications on those machines would now catch the forwarded event exactly as if it were local. The application need not be modifed! This basically implements a Puny WOF (Workgroup Object Facility). And it can be done very easily indeed! Admittedly that this is quite puny and many details about robustness etc. have to be managed. On the other hand, using xnews as a WOF prototyper is extremely appealing. (I plan to prototype a versions of Puny WOF as soon as I get a second host set up and running in my office.) Multi-Media: NeWS Device Control ---------------------------------------------------------------------- Multi-media devices can be managed by NeWS, using programmable events to interface to applications. This provides a logical division between the device "driver", which may be implemented in any of several ways, and the application API to the device, which is via NeWS events. As an example, consider a graphics tablet input device with an RS232 interface. This device emits x,y data in a well-known format to the serial device when the tablet's stylus moves. As an initial implementation, the driver half of the device was written in PostScript as a process which opens and reads the file /dev/tty. When there is no data on the device, the process blocks, pausing until data arives for the device. When the data arives, the read returns with the device data. This is packaged into an event as in the WUE Alarm example. Typically the device data needs processing that is better suited to C programming. The solution is to rewrite the device manipulation as a NeWS client using either CPS or the TNT Wire Service. The application interface, however, continues to be the same event interface. The improved performance incurrs no change in the application interface. The notion of this variety of client "daemon" has been used in TNT for providing shared text service among applicaions. A voice interface to EMacs was also made in AD using the same technique. Another significant advantage to this division of labor into a device layer using an event interface to applications is that it gracefully evolves into sharing the single device among multiple applications. The device is temporarily "owned" by one application, and is given to another application using an agreed upon protocol. Two common styles are used: -The keyboard uses a "focus" model where certain UI events not associated with the keyboard determine which application owns it. -The mouse is typically owned by the application in which it was last "clicked". In the case of the graphics tablet, applications wanting to use the tablet would express interest in the tablet's button being clicked within it. From then on, that application will receive the motion change events for the tablet. Although device ownership may seem like a trivial problem, fairly complex systems are typically needed to solve it for a given device. The VOX server from Olivetti for voice is used for just this kind of thing: negociation between applications for ownership of the voice device. Because VOX is not integrated into a more general event system, however, it cannot manage the *user interface* for which application currently owns it. It can only provide the regestry. When OpenLook defines the "focus" model for ownership of the voice device, a NeWS oriented voice service will much more easily integrate into the rest of the system. Just as WUE Alarms had a network component, certain multi-media devices will be expensive enough to warrent sharing on the network. Scanners and fax machines will be attached to individual machnes, but we may want them to be temporarily owned by other workstations. The same "server" technology used by the 10-line server and the Puny WOF can also be used here. Summary and Vision ---------------------------------------------------------------------- The interesting software architectures of the next decade will leverage network technologies. We will see WUE become the Net Finder, replacing the simplistic Mac Finder in the network intensive workstation world. WUE will manage network resources among WUE-Friendly NetStations, workstations with advanced, easily configured network software platforms. NeWS is such a network tool: a programmable network facility. Another is the newly proposed NFS which uses file server extended PostScript as its client-server language. We hear far too many NeWS vs. X arguments based on window and UI toolkit arguments alone. These, in my opinion, miss the main strengths of NeWS. It is a programmable network facility which can be used for many purposes. It can prototype new network services. It can be a universal device controler. It can be used to integrate multiple hosts. All these elements also need to be evaluated when considering NeWS. References ---------------------------------------------------------------------- [1] PostScript Language Reference Manual, Adobe Systems, Addison Wesley, 1985 [2] The NeWS Book, Gosling & Rosenthal, Springer-Verlag, 1989 [3] Unix Networking-ch 10: Networking NeWS, Owen Densmore, Hayden Books, 1989 [4] NFS-3 Design Document, Brent Calaghan, Sun internal document, 1990 Appendix A: A more general ps2bits shell script. ---------------------------------------------------------------------- #! /bin/sh # Parse cmd line args: ps2bits [-size W H] [-dpi X Y] [-color] X=72; Y=72; W=8.5; H=11; Z=1 while [ ${#} -gt 0 ]; do case "${1}" in -dpi) shift; X=${1}; shift; Y=${1} ;; -size) shift; W=${1}; shift; H=${1} ;; -color) Z=8 ;; esac; shift done # Run NeWS as Unix filter. Note use of Bourne shell variables! xnews " (NeWS/pack.ps) (r) file cvx exec (NeWS/basics.ps) (r) file cvx exec (NeWS/redbook.ps) (r) file cvx exec 500 dict begin false setautobind () stringwidth pop pop $X $W mul $Y $H mul $Z [$X 72 div 0 0 $Y -72 div 0 $H] null buildimage setcanvas (%stdin) run clippath (%stdout) (w) file writecanvas "