The team has investigated issues involved in developing a scalable WWW server on a cluster of workstations and parallel machines, using the Hypertext Transport Protocol (HTTP). The main objective is to improve the processing capabilities of the ADL server by utilizing the power of multicomputers and client resources to match the demands of simultaneous access requests from the WWW.
In previous years of the project, the team developed and implemented a system called SWEB on a distributed memory machine, the Meiko CS-2, and networked SUN and DEC workstations. Each processing unit is a workstation linked to a local disk. The disks are NFS-mounted to all processing units. Scalability of the server is achieved through effective resource utilization by actively monitoring the run-time CPU, disk I/O, network loads of system resource units, dynamically scheduling user HTTP requests to a proper workstation for efficient processing. The distinguishing feature of the scheduling scheme is that it considers the aggregate impact of multiple resource load factors (e.g. CPU, I/O channels and interconnection network) on the choice of processor assignment. Previous work typically considered one resource load factor in the scheduling scheme.
Results indicate that over slow, bus-type networks such as Ethernet, taking into account file location can improve response times in busy server environments by a factor of 50% or more. On faster networks, the effect is still noticeable, at approximately 10%, such as on our Meiko CS-2. Scheduling a heavy load of heterogeneous tasks such as fetching large (10MB) and small (2K) files simulating a busy web site gives a speedup of approximately 50% over standard techniques on the Meiko. Browsing wavelet-based images is also substantially improved.