Keynotes and Invited Talks

Mor Harchol-Balter
Associate Department Head for Graduate Education, Department of Computer Science
Carnegie Mellon University

Server farms are ubiquitous in applications ranging from Web server farms to high-performance supercomputing systems to call centers. The popularity of the server farm architecture is understandable, as it allows for increased performance, while being cost-effective and easily scalable. Given the prevalence of server farms, it is surprising that even at this late date, so little is understood regarding their performance as compared with their single-server counterpart. In this talk, we review existing results and present new results on the stochastic analysis of server farms. We will be particularly interested in the routing/dispatching policies used for assigning jobs to servers, and in the scheduling policies employed at the servers. We will assume that job service requirements are highly-variable, as is common in computing workloads today. We consider three different server farm models. The first model, representative of supercomputing and manufacturing applications, involves non-preemptive, First-Come-First-Serve (FCFS) scheduling at the individual servers. We will see that the mean response time of such FCFS server farms can vary by orders of magnitude depending on the routing policy used, and that routing policies that reduce workload variability are most effective. The second model, representative of Web server farms, employs Processor-Sharing (PS) service order at the servers. We show that the best routing policies for PS server farms are very different from those for FCFS server farms. In particular, the Join-the-Shortest-Queue routing policy will prove to be very useful in the PS server farm setting. Finally, we turn to the question of what server farm architectures are optimal for minimizing mean response time. Here we consider a third model for server farms, where the individual servers employ Shortest-Remaining-Processing-Time (SRPT) scheduling. Such models are very difficult to analyze stochastically, but we discuss some beautiful competitive ratio analyses, that shed light on future research directions for stochastic modeling.

Dr. Ulrich Marquard
SAP

SAP is the world's leading provider of business software. It delivers a comprehensive range of software products and services to its customers: Companies from all types of industries, ranging from small businesses to large, multinational enterprises engaged in global markets. For over 15 years SAP and its hardware/technology partners have developed and used benchmarks to test the performance and scalability of SAP solutions and the hardware they run on. The benchmark results prove the customers that the tested hardware and software configurations can handle their required business load.

This key note will show the principles and methodology behind the SAP Standard Application Benchmarks, the suite of benchmarks developed by SAP and its hardware/technology partners with a view to analyzing the performance and scalability of SAP solutions, and to enable a comparison of the performance of computer systems from different vendors. The SAP Standard Application Benchmarks have established themselves as some of the most credible and popular application benchmarks in the industry, not least of all because they combine the measurement of system performance with a business application that is productively used in customer implementations. A standard process and an independent governing body for the definition, execution, submission and certification of these benchmarks - the SAP Benchmark Council - ensure that the benchmarks are portable across all major platforms and operating systems, and that they generate reproducible and publishable results whose integrity is beyond doubt.

The benefits of the benchmarks and the underlying standardized processes are manifold: The SAP Standard Application Benchmarks help hardware and technology partners optimize their technologies for SAP's applications and enable them to pass the resulting performance gains on to the customers' businesses. Customers also benefit from the strong correlations between the benchmarks and actual customer implementations: by analyzing benchmarking results, available at www.sap.com/benchmark, they can anticipate how a particular hardware and software configuration may behave under high load. Last but not least, SAP itself uses the SAP Standard Application Benchmarks for quality assurance purposes, e.g. to monitor resource consumption during development of a new release, to analyze different system configurations and parameter settings, and to verify hardware sizing.

Murray Woodside
Carleton University, Ottawa

One use of a performance model is to extrapolate beyond the conditions in which performance has been measured. The talk will address extrapolations intended
- to assist in designing system configurations and
- to estimate product scalability,
and will consider the problem of selecting the model structure and estimating its parameters. The problem is analogous to curve-fitting, with the performance model taking the role of a nonlinear fitted function. A more complex model structure (like additional terms in a fitted function) should be judged based on the significance of the reduction in fitting error. Confidence intervals should be found for the parameters. None of this has been worked out in detail for performance models, but an outline will be attempted here. The recent use of sequential filtering estimators for estimating hidden parameters provides some insight, and some techniques that work.

Anil Kumar and Larry D. Gray
Intel Corporation, USA

SPEC has recently released SPECpower_ssj2008, the first industry benchmark which measures performance and power of volume server class computers using graduated load levels. In this presentation, we present a brief overview and an initial characterization of SPECpower_ssj2008 by measuring the system resource utilization with the aid of processor monitoring events at graduated load levels and by comparing the sensitivity of final metric and other related data between various configurations consisting of hardware changes as well as software changes on Quad Core Intel Xeon processor based servers. Even though this is early data from a specific platform and OS, it still validates many expected patterns and opens exciting new opportunities for researchers to investigate specific areas as well as in-depth characterization as a next step.