The ATNF Gigabit Wide-Area Network
CSIRO's Australia Telescope National Facility (ATNF) has recently commissioned 1Gbit/s links to its three observatories at Parkes, Mopra and Narrabri. Although high bandwidth network links are common in metropolitan areas, challenges abound when trying to deliver similar connections in rural and regional locations. Although the primary motivation was to implement a network capable of supporting e-VLBI, the same infrastructure needs to cater for other classes of network traffic such as e-mail, video conferencing and regularly scheduled remote observing. This presentation will discuss the physical and logical design of this network along with prospects and thoughts for future enhancements.
VLBI_UDP has been designed to transfer VLBI data via the UDP protocol, with sustained bandwidth being favored over a 100% bit-perfect stream. User data rates of 957Mbit/s have been achieved over a private gigabit link between Manchester UK and Dwingeloo NL, and 700Mbit/s over production networks between Torun PL and Dwingeloo NL. This report details the development and some results from the application, along with future tests and plans, including correlation tests at various packet loss rates.
The design portion of the National Radio Astronomy Observatory’s Expanded Very Large Array (EVLA) project is approaching completion. Five of the twenty-eight antennas have been upgraded into the final configuration. The 2200 miles of fiber optic cables have been installed underground and are functional. The EVLA is presently a stand-alone system where data only flows from the antennas to the central correlator. The data rate from each antenna is 120 Gigabits, comprised of twelve 10 gigabit channels which are very similar to OC192, and are sent over a single optical fiber. The optical digital transmission system has been transmitting astronomical data for over two years and has provided hundreds of hours of test observations. Link performance has been characterized. This paper discusses the data format, the results of recent testing and the techniques used to maintain link performance.
We report on the progress made by e-VLBI in Europe (the e-EVN) from an astronomical perspective. Since February 2006, the e-EVN has been providing astronomers with open access to an e-VLBI facility that includes 6 antennas, including two of the largest telescopes in the world - the Westerbork Phased Array and 76-m Jodrell Bank Lovell Telescopes. The sustained data transfer rate is currently 128 Mbps (April 2006); correlation occurs in real-time allowing astronomers to quickly access and assess their data. A target of opportunity (ToO) capability is under development. We report on ToO observations of Cyg X-3 during the largest flare ever recorded for this (or any other) galactic continuum radio source. We believe this is the first time that e-VLBI has been used in a rapid response observing mode.
This paper describes the results from some recent tests made to investigate how the network transport layer may affect the behavior of applications. Of particular interest to e-VLBI transfers is how TCP behavior can affect the presentation of constant bit rate data to applications over long distance networks. Several input data rates have been examined with and without controlled packet loss. The aim is to determine how the arrival times of the data at the receiving application vary with the operation of the TCP transport layer. Plots of application behavior using a request-response application protocol over TCP and standard bulk data movement will be shown. For comparison, the behavior and stability of an application sending constant bit rate data using UDP transport will also be presented.
Recent Linux kernels include a infrastructure for pluggable TCP congestion avoidance algorithms. This infrastructure makes it easy to experiment with new congestion avoidance algorithms. We have implemented a kernel module that implements an algorithm similar to C-TCP, a TCP implementation for circuit-switched networks developed by the CHEETAH project. Initial testing of the algorithm in shows an improvement of the sustainable data rate on mildly congested networks. We are currently testing its performance with data transfer from e-EVN telescopes.
By using a short baseline between Kashima 34m and 11m stations, we have succeeded to perform real-time correlation processing with the K5 software correlator program at the data rate of 512Mbps (256Msps, 2bits/sample). Similar experiments are now under preparation between Kashima and Koganei baseline (~100km) over the JGN2 network. After succeeding to perform the experiments, we would like to extend the baseline to international baselines by utilizing GMPLS over the multiple network connections. We will report the current status and future plans for these developments, and would like to discuss about the possibility to perform large scale demonstration experiments in near future.
e-VLBI practitioners have considerable networking power at their fingertips: access to high speed WAN networks with core bandwidths of up to 10 Gbps (potentially even higher), high performance servers capable of driving gigabits per second of data into these networks, and radio telescopes capable of providing data streams at these rates for hours, or even days at a time. While this power may be used to do considerable good (e-VLBI), in the wrong hands, it can do considerable evil (Distributed Denial of Service attacks capable of taking down the most well connected and resourced websites). In this presentation, we discuss some of the security threats to e-VLBI networks, and then discuss some simple steps that can be taken to minimize the chances of an e-VLBI network be taken over and used for nefarious purposes.
What does the ubiquitous availability of 10 Gbps Ethernet technology mean to (e-)VLBI? First experiences of Metsahovi 10 Gbps network connectivity are summarized together with an interpretation of COTS technology roadmap into the near future. Technology and protocol bottlenecks, limitations, and scalability issues are discussed and guidelines for resolving these are presented.
Around year 530 A.D. Emperor Justitianus I ordered that the Roman law was to be written down. The books were called pandects (the all-containing or the rules about everything). At first it was forbidden to make comments about the law. Five hundred years later the glossators and commentators started to do just that. In eVLBI we are bound by the rules and restrictions of tape-age VLBI. If we examine these restrictions closely, we notice that some of these are no longer applicable to eVLBI or can be easily circumvented. A new model for organizing and processing eVLBI data will be presented, with particular consideration for a growth path to future systems.
The Max-Planck-Institute for Radioastronomy at Bonn operates a MK IV VLBI correlator onsite and a 100-m class radiotelescope at a distance of about 45 km from the institute. In December of this year a Lofar station will be erected at the telescope site. To participate in the European and worldwide eVLBI efforts and to connect the new Lofar station to the correlator at Groningen (NL) a fibre connection between the telescope site, the institute and the closest POP of the German academic network (DFN) has to be built. A proposal for a fibre connection to the Max-Planck-Society has been successful. In addition financial support has been promised by the EU. A status report will be given.
The issue of kernel version and distribution for use with Mark 5 systems was discussed at the TOW 2005 workshop. The result was a recommendation and input from various participants at the meeting. Presently, stations are proceeding to experiment with various motherboards, versions of the 2.6 kernel and different distributions. This presentation will focus on a recommendation for an upgrade path for the VLBI community that will satisfy existing users, including e-VLBI users. The presentation will also address issues associated with the Jungo Drivers, Conduant software, and the Mark5 application.
e-VLBI is an international endeavor, and thus requires having data generated in different formats, by various systems, to be handled efficiently and simply as possible. The proposed VSI-E standard, which defines the format of the data on the wire in the network cloud, is a critical component. The standardization of the data format on the line, allows all end stations to process data efficiently, from any source, using only a single conversion to their own data format. This tutorial will again present an overview and a detailed explanation of the VSI-E. We will also discuss lessons learned from testing with real-time and non-real time transfers using VSI-E.
The proposed VSI-E standard, implemented in the reference VLBI Transport Protocol (vtp) application was recently demonstrated with non-real bulk data transfers for R1 and R&D sessions from NyAlesund to Haystack. This presentation will provide the results of these transfers using the vtp. The results show that the VSI-E, using RTP over rate controlled UDP, with a simple packet fill mechanism can meet the needs of non-real time transfers controlled to an upper bound agreed upon with the service providers. Following the trial, another UDP based transfer protocol, a functionally reliable congestion control transport level protocol (VFER), was exercised for comparison and the results provided.
The MIT Haystack Software Suite for e-VLBI real time and non-real time data transfers under alpha testing will be summerized. This is followed by how the software suite is to integrate to the DRAGON project for setting up and tearing down dedicated light paths.
This paper presents an ns-2 based simulation model for e-VLBI traffic and the bottlenecks currently limiting the transfer speeds of astronomical data from radio telescopes across the world over high speed Internet links to the central processing centre in the Netherlands. This model is useful for studying the transport characteristics of e-VLBI data as it traverses the networks. This model accentuates that a combination of large idle times between data bursts caused by application limitation, inefficient receiver hardware and excessive background traffic negatively affect the performance of e-VLBI data transfers.
By even conservative estimates, over the last ten years research networks have increased their capacity and performance by three orders of magnitude. Yet this increase in speed and performance is still unable to satisfy the needs of many emerging e-science application. The global E-VLBI activities are a classic - and recently highly visible - example of this situation. While our core network backbones are able to carry 10s of gigabits per second, the performance of even simple applications such as file transfers are still unable to avail themselves of this capacity end to end, and on a regular, reliable, and predictable basis. A major shift in network architectures is taking place today. For the last 25 years the prevailing notion has been that packet based networks were/are the best answer to data networking requirements. Recent activities however have shown that connection oriented services that provide dedicated and deterministic performance are indeed viable and cost effective means of meeting the needs of a particular set of networked applications. These new connection oriented services - often referred to as "light paths" - integrate into and complement the traditional packet based services to form "hybrid networks" - networks that can provide best of both packet and connection capabilities.
We report on the development of e-VLBI at Jodrell Bank Observatory as part of the ESLEA project. ESLEA is a UK project to exploit the use of switched-lightpath optical networks for various applications, including e-VLBI, HEP, High Performance Computing and e-HEALTH. We show how the CPU power of the Jodrell Bank Mark 5 end systems was limiting the transfer rate to below 512 Mbit/s. Both of the Jodrell Bank Mark 5 end systems have now been upgraded and can now transfer e-VLBI data to JIVE at a data rate of 512 Mbit/s.
With the formal start of the EXPReS proposal (March 2006) e-VLBI developments in Europe have gained momentum. I will report on the current developments at JIVE related to SA1 (production e-VLBI service) and JRA1 (high data-rate e-VLBI and distributed correlation). I will also talk briefly about an NWO-funded project called SCARIe, which will address distributed correlation and lightpath switching on the Dutch Grid.
We report the results and future plan about the geodesy e-VLBI by using the Super-Sinet in Japan. The experiments have been done between Gifu University 11m telescope and the Geographical Survey Institute 32m telescope, the baseline length is about 311km. Two stations are linked with two 2.4Gbps optical networks by the Super-Sinet. More than 10 times geodesy VLBI experiments had been done with the e-VLBI (4Gbps real time processing mode) and K5(128Mbps HDD recording mode) simultaneously, the results are coincided within 3mm. We are planning to up the observation frequency to 22GHz with 4ch (total date rates of 8Gbps) extra wide band mode for geodesy (and also astronomy) e-VLBI.
Since mid-May of 2005, intensive sessions (IVS-INT02) with TSUKUB32-WETTZELL baseline on Saturday and Sunday for UT1 have used high-speed network data transfer. The data which are recorded on Mark 5 at WETTZELL and on K5/VSSP at TSUKUB32 are transferred to Tsukuba VLBI correlator via eVLBI. After WETTZELL Mark5 data are converted to K5 format in a couple of hours, the correlation is processed on K5/VSSP distributed computing. The set of processing is usually completed on Monday. K5/VSSP correlation system can give performance more efficiently in processing eVLBI data. In this presentation, I introduce our contribution to Deep Impact Mission July 2005 with taking advantage of eVLBI intensive session.
In the last few years the Long Baseline Array (LBA) in the Southern hemisphere has been making rapid progress towards e-VLBI. Most recent developments include the completion of the “last-mile” fibre links to many of the telescopes and the development of powerful and versatile software correlators. We will report, on behalf of the LBA team, recent progress and future plans for e-VLBI from a southern hemisphere perspective.
We present general purpose open source FPGA-based hardware and software libraries currently used at 12 observatories for correlators, beam formers, vlbi samplers/filterbanks, spectrometers, RFI mitigation, pulsar timing/searching, antenna holography and SETI. We also present some of our new software tools that make it relatively easy to program FPGA's (even an astronomer can do it!), and how our group built eight instruments in the past two years.
The Mark 5B VLBI data system is designed to support the VSI-H international specification. It is based on the same physical platform and uses the same disk-modules as the Mark 5A; it also supports the same maximum data rate of 1024 Mbps. Data formatting and time-tagging is done internally within the Mark 5B. This allows, for example, existing VLBA systems to bypass the VLBA formatter, which is limited to 512 Mbps, and connect directly to the output of VLBA samplers (through a simple interface) at a maximum data rate of 1024 Mbps. For existing 14-BBC Mark 4 systems, the Mark 5B allows connection of all 14 BBC’s to two Mark 5B’s for a total aggregate data rate of 1792 Mbps. In addition, the Mark 5B is designed to support all critical functionality of the Mark 4 Station Unit, so that the Mark 5B can play back directly to the Mark 4 correlator through a simple interface. The Mark 5B system is currently being used in routine VLBI experiments and is available from Conduant Corp.
e-VLBI technology has been rapidly developed in China. There are four VLBI stations and a VLBI data processing center in China now. The new e-VLBI developments and e-VLBI application results will be reported in this paper.
EXPReS's new project manager will provide an update on plans and activities for EXPReS, touching on current project status/activities and the use of long-distance, advanced networking for e-VLBI.
We wish to create using high-speed communication networks, a distributed, large-scale astronomical instrument of continental and inter-continental dimensions, a Very Long Baseline Interferometer(VLBI) operating in real-time, and connecting together some of the largest and most sensitive radio telescopes on the planet. The overall objective of EXPReS, is to create a production-level 'electronic' VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central supercomputer at JIVE in the Netherlands, via a high-speed optical-fibre communication network, including the pan-European research network, GÉANT. We describe here the status of the last-mile connection of the participating telescopes.
The Onsala Space Observatory is the Swedish National Facility for Radio Astronomy. The observatory operates the two radio telescopes in Onsala, 45 km south of Göteborg, in the mm and cm wavelength regions, and is a partner of the the sub-mm radio telescope APEX at Llano Chajnantor, Chile. During the last years Onsala has played a major role in the testing of real-time e-VLBI in both the astronomical and space-geodesy communities. This has been made possible by the advent of the disk based Mk5 terminals and the construction of a 1Gbit/s optic fibre link from Onsala to Chalmers in 2003. The first e-VLBI milestone during 2004 was the detection of real-time 128Mbit/s fringes between Onsala and two other European antennas in January. This was rapidly followed in March 2004 by historic first real-time e-VLBI fringes across the Atlantic; between the Onsala 20m antenna and the VLBI antenna in Westford, Massachusetts. Interestingly these two stations also gave the first ever tape-based transatlantic fringes in 1968. In September 2005 we installed equipment for coarse wavelength division multiplexing (CWDM, Transmode WDM), separating the Mark5-unit network traffic from the rest of the Onsala Observatory network traffic, This allows us to use the 1 Gbps link with MTU sizes up to 4470 with the Mark5-unit while the observatory network uses a MTU size of 1500. On our poster we present our involvement in current and future applications of eVLBI and the optical fibre connection form the Onsala Space Observatory via SuNet/NordUnet to international research networks.
The LBADR is a COTS VLBI recorded based on the Metsähovi Radio Observatory VSIB digital input card. This system used standard computer disks for recording based band data up to 512 Mbps either locally or via the network. This poser will present the architecture of the LBADR system and highlite its flexibility.
In this presentation, we introduce the development of Korea-Japan Joint VLBI Correlator(KJJVC) and KVN(Korean VLBI Network) construction briefly. The KVN is now building 3 new high-precision radio-telescopes of 21-diameters in 3 places (Seoul, Ulsan, Jeju). It is the dedicated for mm-wave VLBI observation and the first VLBI facility in Korea. And we are also developing the KJJVC for the combined VLBI network consisting of KVN and VERA including several other telescopes. It is being discussed mainly by the Korean and Japanese correlator teams and applied several new functions and ideas. We present some recent status from both KJJVC development and KVN construction.
TIGO, located in Chile, is one of the few stations of the VLBI network in the southern hemisphere which have a limited international bandwidth restricting the incorporation to the eVLBI community. Recent speed tests, framed by SMART-1 event, have shown a maximum speed of 5 Mbps to international sites. There are two approaches under research to solve this issue which are detailed in the present poster. The first one, the ExPRESS project, aims to identify the bottlenecks and then upgrade links and/or equipments where it’s required, while the second one, the Multipath Routing, looks for a cooperative solution where the bandwidth is the sharing resource involved.
A poster detailing eVLBI Production at Haystack Observatory. Displaying details of statistics, implementation, overall process, developmental outlook, and more.
We present details of a software correlator developed for VLBI and real-time eVLBI. The software has been developed for multi-processor computing environments and uses standard libraries for signal processing. The correlator is now part of the Australian Long baseline Array and has been in routine use for almost 12 months. Over the next 12 months all Australian VLBI will migrate to disk-based recording and software corrlelation, from the current S2 system. We present some recent results from both VLBI and eVLBI experiments.
We report the feasibility of Earth orientation calibration using DSN network enabled e-VLBI. The current operational DSN VLBI calibration of Earth orientation parameters (EOP) is based on magnetic tape recording and usually has ~ 7 days of delay (turnaround) in delivering EOPs to the JPL Kalman Earth Orientation Filter (KEOF). The current limitation in real-time knowledge of UT1 is the turnaround time of tape-based VLBI. Basic elements of the e-VLBI system include digital recording instrument (Wideband VLBI Science Receiver or, WVSR), DSN network transfer of raw data, software correlator (Softc), post-correlation analysis software. The goal is to achieve UT1 knowledge better than the current operational system. With improved network transfer rate, a shorter turnaround time of calibration delivery is promised. With data latency being the dominant error source in UT1 series, quicker turnaround means better UT1 real time knowledge.
Return to E-VLBI Meeting Page