Internet2 Fellowship on Application QoS Needs

The purpose of this document is to describe an Internet2 Fellowship to study and document applications QoS needs. Specifically, the recipient of this fellowship will produce a survey paper and create a framework for the ongoing analysis and documentation of the relationship between the needs of advanced Internet2 applications and new network services being deployed by Internet2.

This is a Fall(2001-02)/short-term fellowship, and only one grant is available. Applicants can be masters or doctoral students or faculty members in computer science or a related field. Candidates will be evaluated on the basis of their scholarly achievement and course work, research, or teaching in areas relevant to IP quality of service. The applicant must be available for six months in order to complete the expected document. The fellowship will pay $10,000.

Status: fellowship has been awarded to Dimitros Miras, PhD student from University College London, UK.

Goal

The primary goal of this effort is to provide a focal point for activities aimed at fostering a better understanding of the network demands of new applications, how those applications may exploit new network services, and at what cost and benefit. Specifically, applications should be evaluated in the context of three Internet2 network services: 1) "typical" lightly-loaded Internet2 best effort service; 2) the QBone Premium Service (a virtual circuit service with strong bounds on loss and jitter); and 3) the QBone Scavenger Service (a bottom-feeding class of best effort).

In particular, the expected output of this Fellowship is a survey that includes the following:

1. Taxonomy of advanced Internet2 applications:
   The document is supposed to classify different types of Advanced Internet2 Applications, i.e. the applications that can benefit from the service differentiation over the Internet2. It should also provide the basic information about those applications and point to the sources with detailed information.

2. Application behavior:
   The author should identify application behavior in terms of the fashion in which network services and resources are required. Some applications provide a constant flow of data and therefore will pose a demand for constant provision of network resources, while some others will pose occasional but high-pitched demands. Such different behaviors could have an important impact on the way the demands for networking resources are processed and reserved.

3. Quantitative measure of application needs:
   The document should contain a review of the studies done so far that have treated application needs for network services in quantitative terms (e.g. loss, delay and jitter tolerances). Such parameters represent desirable/minimal/maximal values that have to be achieved for different sensory and control data (video, audio, haptic, tracking, database and event transactions, simulation, remote rendering, control, etc.) so as to classify system response as "real-time". In many applications the lack of such responses causes the break in presence and copresence, which has a direct impact on perceived quality of the session on application level.

4. Tools and experimentation procedures:
   Finally, a very important part of the document should focus on the tools and experimental measurements procedures that are available or are being developed for quantifying how different levels of service and resource guarantees in the network actually translate into application level quality improvements. Such tools are critical to the successful deployment and usage of service differentiation in the Internet2, as they allow the parameterization of how increasing network resources or guarantees affects application or user level utility. We regard this as the true measure of interest when assessing the benefits of service differentiation. Providing such an assessment is key to allowing both network administrators and application users and developers to determine when and how to use Internet2 service differentiation capabilities. The document should also take into account the efforts and the results of other Internet2 initiatives that are closely related to the topic of this Fellowship (End-to-end Initiative, Measurement Working Group).

Because new activities, applications, studies and tools will be developed over time, the survey produced by the recipient of this fellowship is expected to be a living document that should be structured and formatted to allow easy updates. Indeed, it is hoped that the survey could serve as a framework for the ongoing analysis, evaluation, and documentation of additional Internet2 applications and/or network services.

The Fellowship will have assigned advisory committee made of three members; the committee will provide the interactions and professional support needed for the work. The holder of the Fellowship will be requested to provide regular incremental information about the progress with the work (once in two weeks) and monthly reports on the current status of the document.

Additional Reference Information:

QUANTITATIVE MEASURE OF APPLICATION NEEDS:

Presence is usually defined as a psychological sense of being part of an environment that is different from your physical environment, and copresence (or social presence) as a sense of sharing such place with other people. In case of advanced Internet2 applications those factors are extremely important for undisturbed communication flow and interaction among the remotely coupled users as well as between users and the system. Due to the multidimensional nature of sensory inputs and a variety of psychological responses that they produce it is very difficult to get precise metrics and measure such phenomena. However, the practitioners in this area produced numerous empirical studies on how different networking parameters impact people's perception as well as their task performance, and they are valuable pointers for better understanding of the requirements that will be required from the network. The results of related studies can be found in [1], [2], [3], [4], [5], [6] to name just a few. An extensive bibliography and list of related links can be found at [7] and [8].

The document should give a survey of measurements techniques used, summary of the quantitative values of the measured parameters reported in the studies, and underline the dependencies between measured parameters and network resources wherever such correlation exists.

References:

[1]

Barfield, W., & Hendrix, C., "The effect of update rate on the sense of presence within virtual environments", Virtual Reality: The Journal of the Virtual Reality Society, 1(1), 3-16, 1995

[2]

Barfield, W., Baird, K. M., & Bjorneseth, O. J., "Presence in virtual environments as a function of type of input device and display update rate", Displays, 19(2), 91-98, 1998

[3]

Welch, R. B., Blackmon, T. T., Liu, A., Mellers, B. A., & Stark, L. W., "The effects of pictorial realism, delay of visual feedback, and observer interactivity on the subjective sense of presence", Presence: Teleoperators and Virtual Environments, 5(3), 263-273, 1996

[4]

Slater, M., & Wilbur, S., "A framework for immersive virtual environments (five): speculations on the role of presence in virtual environments", Presence-Teleoperators and Virtual Environments, 6(6), 603-616, 1997

[5]

Rick Stevens, "Steps Toward the Access Grid", presentation given at the Access Grid at SC'99, Portland, Oregon - November 15-19, 1999 http://www-fp.mcs.anl.gov/~stevens/all-presentations/capstone-vis99/sld001.htm

[6]

Tromp, J., Steed, A., Frecon, E. Bullock, A., Sadagic, A., Slater, M., "Small Group Behaviour in the COVEN Project", IEEE Computer Graphics and Applications, 18(6), 53-63, 1998

[7]

http://www.hitl.washingtonton.edu/publications/r-99-9/

[8]

http://nimbus.temple.edu/~mlombard/Presence/

TOOLS

The expected output of the Fellowship is to include both a listing of applications for which service differentiation has been identified as a potentially important enabler, and for each of them a survey and description of quality assessment tools. Note that most such tools are likely to be application specific, and it is also most likely that for some applications there will not be any such tool available. The survey should identify relevant existing tools, explain their general characteristics and capabilities, provide pointers to additional information (publications, technical reports, standard documents, ftp sites for source code, etc.), and point to potential limitations of the tools in the context of their application to Internet2 applications as well as any ongoing effort to address those. In case such tools are not available the survey should provide suggestions for functionalities that would be desirable to exist in appropriate new tools.

For example, both audio and video streaming applications are expected to derive benefits from service differentiation when transmitted over the Internet2, i.e., in the form of delay and/or bandwidth guarantees such as those provided by the Abilene Premium Service. However, the extent of those benefits and for which level of resources they are attained remains unclear, and it is, therefore, of interest to understand the coupling that exists between the two and to identify tools that might be available to perform such a task. Although such tools are not readily available, there has been a significant level of related work aimed at quality assessment of voice and video signals in more traditional environments. For example, objective and subjective video quality assessment methods for television and video conferencing technologies have been extensively investigated and even standardized through standard bodies such as ANSI and ITU (see [1-6] below), and some tools have been developed that are capable of performing such assessments.

One such example is the ITS Video Quality Measurement (VQM) software tool of [5]. The tool is based on a family of objective quality assessment methods called Feature Extraction or Reduced Reference. Those methods rely on mathematical models to capture the major features of either individual frames (spatial features) or sequence of frames (temporal features) from both the received video stream and the reference video stream. The quality of a received series of video frames is then assessed by comparing the time histories of the received feature streams with the reference feature streams, and by combining multiple quality parameters so generated into an overall quality score. This is only but one sample of several possible tools that might be of relevance and adapted to allow subjective measurement of video streaming quality over the Internet2 for different levels of service configuration and resource guarantees.

In case of computer graphics, virtual reality and haptics systems similar tools are quite difficult to devise. The major obstacle lies in the fact that these systems have to deal with the plethora of parameters and all of them have an impact on overall perceived quality of the session: multidimensional sensory inputs and the range of sensory modalities accommodated, user interfaces and interactive part of such systems, as well as the task and the level of involvement that particular task induces. In those systems the tools that have been suggested are rather in the form of questionnaires that are given to the users and their responses analyzed afterwards. Typical examples of such questionnaires can be found in [7], [8], [9].

References

[1]

ANSI T1.801.03 - 1996, "American National Standard for Telecommunications - Digital Transport of One-Way Video Signals - Parameters for Objective Performance Assessment", American National Standards Institute.

[2]

ITU-R Recommendation BT.500, "Methodology for subjective assessment of the quality of television pictures", Recommendations of the ITU, Radiocommunication Sector.

[3]

ITU-T Recommendation P.910, "Subjective video quality assessment methods for multimedia applications", Recommendations of the ITU, Telecommunication Standardization Sector.

[4]

ITU-T Recommendation J.143, "User requirements for objective perceptual video quality measurements in digital cable television", Recommendations of the ITU, Telecommunication Standardization Sector.

[5]

S. Wolf and M. H. Pinson, "Spatial-Temporal Distortion Metrics for In-Service Quality Monitoring of Any Digital Video System", Proceedings of SPIE International Symposium on Voice, Video, and Data Communications, Boston, MA, September 11-22, 1999.

[6]

M. Knee, "The Picture Appraisal Rating (PAR) - a single-ended picture quality measure for MPEG-2", Proceedings of the International Broadcasting Convention, Amsterdam, The Netherlands, September 2000.

[7]

Bob G. Witmer and Michael J. Singer, "Measuring Presence in Virtual Environments: A Presence Questionnaire", Presence-Teleoperators and Virtual Environments, Volume 7, Number 3, June 1998

[8]

Mel Slater, "Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire", Presence-Teleoperators and Virtual Environments, Volume 8 Issue 5, October 1999 pp, 560-565

[9]

Roy S. Kalawsky, "VRSART A Tool For Evaluation of Contributory Factors Associated with Presence in Spatially Immersive Environments", Proceedings of the Presence in Shared Virtual Environments Workshop, First International Workshop on Presence, Ipswich, Suffolk, UK, http://sgi-hursk.lut.ac.uk/~avrrc/presence/vrsart.htm