Semester Project Final Report:
Sockets And Data Transfer With JavaBeans

Paddy McCarthy
CSCI 4830
4 May, 1998

Introduction

When building software systems with a data-centered architecture, one of the least general portions of the system is the data-gathering software, which takes care of collecting data from various sources and populating the database on a schedule determined by the needs of the system.

At a high level, this software consists of two parts: 1) A notification system that determines when/where new data is available, and 2) A data transfer mechanism that takes care of sending the data to the database site. This project focuses only on the transfer portion of the problem.

The design of each data gathering system must be different to allow for variations in the timing of data collection, the requirements for data availability on the server, and security requirements and network impediments, such as firewalls. A general, configurable, transfer application is not appropriate, for it would have to contain far too much flexibility to perform a relatively simple task as desired. For this reason, tools which allow the quick construction of networked data transfer applications would be extremely useful. Such tools should provide a suite of transfer components from which the assembler to choose those appropriate to meet the needs of each specific application.

The intent of this project was to design and implement a small set of data transfer beans to determine if this approach would be useful for facilitating the rapid assembly of customized data-trading applications. The components I created were a socket server, a simple socket, a file reader, and a file writer. In addition, I created a component called a DataSwapper that transfers data from one component to another within an application.

In the course of implementing and testing these components, I spent most of my effort trying to find an environment in which the Java security model allowed socket connections. As a result, the focus of the project shifted somewhat from design and implementation to learning about the subtleties involved in trying to work with the security features in Java. The implementation that resulted from my efforts was a set of very bare-bones beans that I assembled into data transfer applications for testing. The implementation did not include any sort of scheduling or coordination components, so all the testing applications required user input to initiate transfers of data.

The resulting solution is very promising. The beans I implemented are basically providers and users of InputStream and OutputStream objects. The primary communication between the components are requests for streams, notifications of available streams and notifications that streams are no longer needed. The primary obstacles to using components such as these to construct applications are implementing a more complete set of transfer beans, making the beans more flexible, and finding a good builder tool which allows the assembler to create applications which can make unrestricted socket connections.

Related Work

Data gathering systems with which I am familiar operate under one of three paradigms. The first, which will be referred to here as polling, involves the data server going out to each data gathering site at some scheduled interval and checking if new data is available, then fetching any new data that is found. The benefits of polling are security, since the server never receives any unsolicited communication from external sources, and a built-in mechanism for determining when data has not arrived on time. Also, there is no notification scheme needed because everything is initiated by the server itself. The drawbacks of this approach is that the server must have knowledge of all of the data collection sites and their associated collection schedules, which is not necessary in systems with a less-structured layout.

The next general method of data collection involves some sort of pinging mechanism in which the data gathering site notifies the server that new data is available at that site. The server, having received notification of new data, can determine if that data is needed and retrieve the data from the collection site. Pinging is good from a security point of view because only an unsolicited message is received by the server, leaving the actual data transfer as a fetch operation performed by the server. Data transfer is decoupled from the notification, which could be secured by encryption, etc.

The third collection method involves each data collection site sending data directly to the data server. This client-initiated transfer method is relatively insecure because many remote machines are connecting with the server and, unsolicited, sending large amounts of data to it. The only system I know which uses this method accepts the data on a conditional basis, subject to human verification. One benefit of client-initiated transfer is that a notification method is no longer needed, as the transfer itself comprises the message that new data is available.There are many possible solutions to solving the data transfer problem described here. The three benefits offered by the proposed solution are security, simplicity, and the opportunity to focus on component implementation.

The examples of transfer applications that I have encountered are all custom-built systems implemented in C or C++. They are implemented for a specific platform and are not broadly reusable. A component solution would provide two advantages: platform independence and easy assembly of the systems from shared components.

Security

Security is extremely important to consider in order to retain the possibility that a transfer mechanism using the proposed work might be implemented on an operational system. There is a recurring need for applications such as this at the Center for Advanced Decision Support in Water and Engineering Systems (CADSWES), where I work. But the platforms where these applications must run have various firewall and administrative constraints to enforce security. Sockets are both relatively secure (with the possibility of using password verification), and often allowed as a means to communicate through firewalls. The State of Colorado, for example, has a firewall on its network that prohibits connections to sockets with IDs <= 1024, which excludes common networking utilities such as ftp and telnet, but allows connections to sockets with higher IDs. Transfer applications for these systems use password verification to enforce security constraints.

Mechanisms

Server-driven transfer systems, which can be employed in a pinging-based or polling-based architecture, can use one of many mechanisms to transfer data or files between systems:

Client-driven transfer systems can use one of the following mechanisms for transferring the data:

Other Solutions

Solutions which employ various distributed computing technologies are unnecessarily complex to solve this problem, and are relatively insecure. Java's Remote Method Invocation (RMI) 1 , and other packages such as Java Party 2 and ObjectSpace's Voyager 3 all provide distributed computing capabilities complete with serialization of objects, but they all seem to rely on the Java security model to keep remote systems from performing illegal operations on the local system. Given the immaturity of Java and the assumption in the java security model that all the interfaces between the java byte code and the OS and hardware are bug-free, government agencies with which we work seem unwilling to forego their security practices of the past. It will be much easier to convince them that locally-run java code using socket connections is secure. In addition, the distributed computing packages mentioned above provide much more flexibility and functionality than is needed to solve the problem at hand. There would not be much gain for the large overhead in complexity.

A CORBA solution to the data transfer problem is probably a closer match to the requirements than true distributed computing technology (CORBA does not support passing of objects). However, I chose not to pursue this option based on the fact that it presented the potential for much higher administrative overhead; I wanted to avoid spending the semester configuring a CORBA server.

Project Description

The solution for this project involved implementing five Java beans and assembling them into simple data transfer applications. I tried many approaches to create applications and applets composed of these beans, but continually ran into obstacles with the Java security model and the builder tools I was using to assemble the components. In the end I gave up and ran the applications directly from the BeanBox, since this class seems to circumvent all the security restrictions. I am unhappy with the fact that I was unable to find a way to use these components in an applet or stand-alone application, but I had to move on from that problem so that I could have some time to work on the bean implementations.

Due to the intense effort involved in figuring out how to avoid Security exceptions when running applications that make socket connections, the implementation of this project is not very elaborate. I implemented the minimum set of components to prove the concept, and each component is intended for a specific use. However, the design for the communication between the components is potentially sound enough to support a larger set of more-flexible beans.

Bean Description And Use

This project involved creating five beans using Java, and assembling them into two applications that transfer data through sockets.

Requirements

The implementation associated with this project shrank due to a need to spend increased effort on exploring different builder tools and Security manager functionality. As this happened, the requirements changed from specifying what each bean needed to provide to a more general requirement that the set of beans together provide enough functionality for proof of concept. The requirements became:

Fortunately I had done significant design work before hitting the implementation crunch, so I was able to take the generality requirement to heart. This requirement is met primarily in the design for how the components communicate.

Bean Design

Five beans comprise the set of components for building the transfer applications. A description of each follows.

Event Design

The events generated by the beans correspond to the methods in the IStreamListener and OStreamListener interfaces. Each bean allows registration of listeners that implement one or both of these interfaces.

Bean Communication

Beans communicate by sending events defined in the IStreamListener and OStreamListener interfaces. Figures 1 and 2 show object sequence diagrams for two sample applications using the beans described above. The diagrams are drawn using an attempt at Unified Modeling Language (UML), which has substantial documentation on the web 4 .

The sequence of events is initiated by the user pressing a button. This event calls the processSwap() method on the DataSwapper bean. This bean then requests an InputStream and an OutputStream from its listeners and waits until they are both obtained. When both an InputStream and OutputStream are received (by listening for gotIStream() and gotOStream() events), the DataSwapper reads text from the InputStream and writes it to the OutputStream until encountering the text "bye". At this point the DataSwapper generates doneWithIStream() and doneWithOStream() events that allow the listeners to free system resources.

 

 

Assembly And Use Of Beans

To create an application with the provided components, you need a bean that generates a user event, such as a Button, a DataSwapper bean, a bean that provides an OutputStream (OugoingSocket or FileWriterBean), and a bean that provides an InputStream (IncomingSocket or FileReaderBean). Drag one of each of these beans off the palette and make the following connections:

Click on each of the beans and set any properties that need to be configured. The following beans have properties that can be set:

Software Process

The software process for this project was complete chaos. I started out with a pretty good set of requirements from which I created the design I presented in my 12 March 1998 Status Report. I drew diagrams of this design by hand using a mix of OMT 5 and Booch 6 notations. I was intending to formalize the requirements at about the time that I felt inclined to do a test implementation to head off any major roadblocks. At that time I encountered my first frustration with the Java security model which continued for a solid four weeks. I spent all my resources figuring out a solution to the security problems, and software process suffered.

I think it's worth mentioning that it's very hard to maintain a formal software process in the face of the unknowns and lack of support that accompany immature technology. Spending time on formalizing requirements, design, implementation, integration, and testing doesn't make any sense if an unexpected roadblock exists that prevents integration. It makes me wonder if the move towards software "engineering" was to some degree made possible by using languages and development environments that were more or less known quantities. The formalization of the software development process may have to backstep somewhat when new technologies and new paradigms for programming are introduced.

The immature quality of the Java technology also affects the size of the increments in which I develop. During MP2 for this course I first did enough work with beans to get a sense for how they work, then coded up a large portion of my solution. I had to undo all that work and come up with a new design because a bug in the BDK didn't allow the serialization of arrays of objects. For this project I never took that risk -- I developed in very very small increments and fully tested each addition by assembling the changed components into an application. This approach is very slow, but it's faster in the long run if a problem arises.

Java Experiences And Frustrations

My struggles with Java during this project were primarily associated with the security model, as I was never able to create an applet or stand-alone application that included my socket beans. Instead, I ran my applications directly from the BeanBox, because it circumvents the security model limitations.

The following items enumerate my attempts to create applications which include the socket beans, and the reasons I gave up on each:

This approach does not work because applets are not allowed to connect with any host other than that from which the applet was downloaded.

This approach allowed me to run an applet which included an IncomingSocket bean. But as soon as a client connected to the port where the IncomingSocket was waiting, the applet exited with the exception: "sun.applet.AppletSecurityException: checkconnect.networkhost1." This is related to the relationship between applets and network connections mentioned above.

I tried at least two different ways of converting applets generated from the BeanBox to applications. Both of them failed with the following exception: "java.net.MalformedURLException: systemresource:/ZIP5/+/: java.lang.SecurityException: systemresource:/ZIP5/+/ refers to a non system resource." My impression is that the BeanBox does something fancy when it creates applets that does not lend them to conversion to applications.

The two approaches I tried for converting the applets to applications were: 1) adding a main() method to the applet, and 2) wrapping the applet in an extension class that instantiated the applet within a frame 7 .

This attempt represents many long hours that ended in a dead end that I suspect is purely due to encountering something that has not yet been implemented. I downloaded Netscape's Capabilities API from their web site and embedded code in my applet to enable the "UniversalConnect" privilege. I ran my applet through Netscape, and when the applet was loaded a dialog appeared to prompt me whether I wanted to enable this privilege. I responded positively and the applet failed with a security exception.

I suspect that this privilege is not yet implemented in the Capabilities API, but I couldn't find any information on the web about it. People I have spoken to have used the Capabilities API to successfully allow applets to write to local disk.

I downloaded HotJava 1.1.2 and looked into using the security manager to allow my applets to connect to remote hosts. The only way to do this seems to involve using signed applets, for which HotJava allows you to enable certain privileges. I spoke with people who had spent a lot of time trying to get a universal signature for some of their own applets, and abandoned this approach.

I downloaded Java-WorkShop2.0 from the SunSoft site and tried to use it to create some stand-alone applications with my beans. This worked fine until I went to connect two beans together in the GUI builder. At that point I discovered that Java Workshop uses some Paleozoic event model to connect components. To make the connection I desired, I would have had to call a method on a global "group" object in the application. I would also have had to register all my components with the group object. From the group object I would have to look up the bean that I was trying to send a message to, and call a method on it. None of this is supported in the GUI builder -- I would have had to implement it by writing code. This was such a radical departure from my goal to "rapidly develop custom applications" that I gave up on this approach.

Had I more time, I would have done more research on GUI builders that support the BeanBox-style assembly of components. Oracle's AppBuilder apparently has a nice interface for this kind of assembly, but it is very expensive. SuperMojo has some sort of support for this, but I downloaded a trial copy of the software from Penumbra's site and had little luck exploring its functionality. SuperMojo is written all in Java, but for some reason runs only in Windows (?), and it died three times on me before I gave up and decided to keep my $99.99 in my pocket.

New Skills And Knowledge

This project provided great opportunity to learn new skills and gain knowledge of Java. These range from practical, concrete skills to very abstract skills in learning new languages and environments.

The most high-level personal benefit from this project was the reminder that learning languages/development tools can be a real struggle. Though this idea isn't new to me, I have definitely gotten used to knowing the details of my primary programming language (C++) and have a well-established support structure for when I run into difficulties. I have a new appreciation for how easily and why people get entrenched. It's much more challenging to work with a language that is still in its formative stages, looking at tools to quickly assess whether they are suitable for an intended project. At a minimum these unknowns have to be factored in to estimates for accomplishing the intended project, something I failed to do.

On a more applied level, I finally learned portions of UML to draw the figures in this summary. I had used UML documents to understand designs before, but had never constructed them myself. I made the effort to learn this symbolic language because I was struggling to find a way to represent the interactions between beans in my applications. UML seems awkward to use with threads, because the notion of a single control no longer exists. It was fine for my relatively simple sequence diagrams, but in complex systems I don't think it would be very helpful.

I also gained a much more in-depth familiarity with Java beans. After completing MP2, I had a basic knowledge of the bean model, but this project solidified that knowledge. Now I have a much more intuitive feel for how to design event sets that make sense, and for when threads are necessary.

This project rejuvenated my faith in the web as a source for information to solve problems. In the past my success with using the web to solve problems has been about 50% or less. I mostly resorted to news groups and list servers for this kind of information. But given the broad interest in Java and the rate at which information is being added to the web, I had much more success during this project.

Finally, I learned a huge amount about the Java security model, the intents for tools that provide greater flexibility within this model, and just how many of them are not implemented yet. See the previous section "Java Experiences And Frustrations" for more details on this.

Proposed Improvements

Given the somewhat skeletal implementation that came from this project, there are many things that could be improved. But two improvements come immediately to mind because they were on my wish list, so the design of the beans would easily allow them.

First, the IncomingSocket and OutgoingSocket should be generalized to allow bidirectional data transfer. This would result in two different beans, a SocketClient and a SocketServer, which could each provide both an InputStream and an OutputStream. In the current implementation, the OutgoingSocket is a SocketClient arbitrarily limited to providing an OutputStream. The IncomingSocket is a SocketServer that only produces an InputStream. There is no reason for these restrictions, except lack of resources to implement the more general solution.

Second, the DataSwapper should be generalized so that user action is not needed to initiate a data transfer. I can see a total of three paradigms that the DataSwapper could operate under:

This is the current implementation. The user must press a button to initiate a transfer. The DataSwapper then requests an InputStream and an OutputStream and performs the swap when both are acquired. When the transfer is complete the DataSwapper waits for user action again to initiate another transfer.

This would be very useful for the IncomingSocket transfer bean. In this case, the DataSwapper would ask for an InputStream on start-up, and wait for that InputStream to be acquired. When the InputStream became available, it would request an OutputStream and transfer the data.

This would be very useful for the IncomingSocket transfer bean when it is generalized to be a SocketServer. In this case, the DataSwapper would ask for an OutputStream on start-up, and wait for that OutputStream to be acquired. When the OutputStream became available, it would request an InputStream and transfer the data.

Evaluation / Conclusions

The initial goals of this project were not met because of problems finding an enviroment in which I could run applications with unrestricted network access. Unfortunately I didn't get as far as I would have liked on a full implementation of transfer beans.

Nor did I never did find a GUI builder that could assemble beans in a style similar to the BeanBox and generate stand-alone applications. That piece of work still remains to be done. All I have provided here is a list of approaches that did not work in April 1998. Hopefully mamy of the shortcomings of the Java environments will be overcome in the near future.

However, I got far enough into the details of implementing some beans to see that the idea is still a good one. I am still encouraged about the applicability and usefulness of a suite of data transfer beans that could be assembled into small transfer applications.

Grading

This text is extracted from the project proposal, with modifications in bold ("NOTE: ...")where I suggest the grading criteria should be modified.

20% Implementation Of Proposed JavaBeans Components

10% Implementation Of Proposed Data Transfer Applications Or Simple Test Application Using New JavaBeans

70% Written Final Project Summary

10% Technical Documentation Describing Functionality And Use Of Beans.

10% Software Process Description -- Design Process And Tools, How Design And Implementation Fit Together, Proposed Improvements For Future Projects.

20% Enumeration Of New Skills Acquired Through Completing This Project.

20% Specific Statement Regarding Applicability Of Socket Components With Generalized Interface.
NOTE: I would like to replace this item with a detailed description of obstacles and attempted solutions for getting the sockets to work within the security model. This seems more useful to the intended readers.

10% New Insights On How Work Might Be Improved, How Java Tools Could Be Improved To Make Implementation Easier, And My Personal Reaction To Working With Beans.

Bibliography

Booch, Grady, 1994, Object-Oriented Analysis and Design, Addison-Wesly, 589pp.

A text describing Booch's notation and methodology for designing and expressing oo relationships. Booch's notation is one of the building blocks of Unified Modeling Language (UML).

Clip, Paul, "Java's Object-Oriented Communications", Byte, vol. 22, no.2, Feb. 1998, pp53-54.

Flanagan, David, 1997, Java In A Nutshell, O'Reilly and Associates, Inc., 610pp.

Basic java language user's bible. Very lean on examples, as these are published in Java Examples In A Nutshell, which is out of print.

Flanagan, David, 1996, Java In A Nutshell, O'Reilly and Associates, Inc., 610pp.

The 1.0 version of the Nutshell book, from which I gleaned the example of instantiating an applet in a Frame object so it can be run as a stand-alone executable.

Horstmann, K.S., and Cornell, G., 1998, Core Java 1.1, Volume 2 - Advanced Features, Sun Microsystems, Inc., pp. 132-185

This is a very useful book. Examples are well-developed and well-explained. My IncomingSocket implementation contains excerpts from examples provided in the "Networking" chapter.

ObjectSpace, Inc., Voyager Users Guide, Version 1.0 Beta 3.0, July 1997.

User's guide for a development environment being used by Richard Hall, a Ph.D. student here in CS who is doing work with distributed computing. I looked at this as an alternative paradigm for data transfer in the nascent stages of the project.

Philippsen, Michael, and Zenger, Matthais, "JavaParty - transparent remote objects in Java", Concurrency: Practice, and Experience, vol. 9, no. 11, Nov. 1997, pp 1225-1242.

Journal article on an RMI-type implementation for Java. I looked at this when I was looking at alternatives to a Java / BeanBox solution.

Rational Software Corporation, http://www.rational.com/, or go directly to the documentation at http://www-dse.doc.ic.ac.uk/local/uml/uml.html

Web site for obtaining documentation on Unified Modeling Language (UML). Portions of UML were used in object sequence diagrams in this document.

Rumbaugh, J., Blaha, M., Pemberlani, W., Eddy, F., and Lorenson, W., 1991, Object-Oriented Modeling and Design, Prentice Hall, 500pp.

A text describing OMT - one methodology and notation for expressing OO design concepts and constructs. I use a mix of this notation and Booch notation for drawing static object diagrams in my daily work.

Sridharan, Prashant, 1997, Java Beans Developer's Resource, Prentice Hall, 353pp.

A very frustrating book, in the fact that it tends to deal with most of the material on a conceptual level but explains it very badly. Small examples lacking depth are spread over ten pages, with most of the intervening text being natural language expression of the code which doesn't add anything. Full of text referring to figures as if they explain themselves, and figures with headings that seem to indicate the text will explain them. Displays consistently terrible grammar. Contains a CD-ROM which holds a version of Super Mojo which does nothing, and enjoyable snippets to enlighten you on all the yuppie favorites: BMWs, coffee, and even a Tiramisu recipe!

Watson, Mark, 1998, Creating Java Beans, Components For Distributed Applications, Morgan Kaufman Publishers, Inc., 237pp, with CD-ROM.

An implementer's book with solid, well-developed examples of high enough complexity to address many of the subtleties needed to make your own code work. Some sections (e.g. see section 5.6 an page 95 - "Testing the SendMail JavaBean") have the appearance of never having been written, so the placeholder text is still in the published version. Examples are nicely broken down into requirements, design, and implementation.


1

Clip, Paul, "Java's Object-Oriented Communications", Byte, vol. 22, no.2, Feb. 1998, pp53-54.

2

Philippsen, Michael, and Zenger, Matthais, "JavaParty - transparent remote objects in Java", Concurrency: Practice, and Experience, vol. 9, no. 11, Nov. 1997, pp 1225-1242.

3

ObjectSpace, Inc., Voyager Users Guide, Version 1.0 Beta 3.0, July 1997.

4

Rational Software Corporation, http://www.rational.com/, or go directly to the documentation at http://www-dse.doc.ic.ac.uk/local/uml/uml.html

5

Rumbaugh, J., Blaha, M., Pemberlani, W., Eddy, F., and Lorenson, W., 1991, Object-Oriented Modeling and Design, Prentice Hall, 500pp.

6

Booch, Grady, 1994, Object-Oriented Analysis and Design, Addison-Wesly, 589pp.

7

Flanagan, David, 1996, Java in a Nutshell, O'Reilly and Associates, Inc., pp101-102. Note that this is the 1st edition Nutshell book for Java 1.0