Substantial progress has been achieved in all three development areas of the VLBA 4 Gbps data path expansion, the major component of the VLBA Sensitivity Upgrade Project. An overview of the 4 Gbps project appeared in Issue 111 of this Newsletter, and detailed reports on specific sub-projects appeared in Issues 112 and 113. This article is an update on recent progress. Ongoing documentation on the entire VLBA Sensitivity Upgrade Project is available in the project memo series.
Work on the Digital Backend (DBE) sub-band processor module has concentrated on final design and layout of the ROACH (Reconfigurable Open Architecture for Computing Hardware; formerly known as “iBOB-2”) board that will serve as the hardware platform for the DBE. ROACH is a further development of the UC Berkeley CASPER laboratory’s very successful iBOB. This device includes a new, high-capacity Xilinx Virtex-5 FPGA, a PowerPC control processor, and a variety of appropriate I/O interfaces. In particular, a 10G Ethernet output interface will support separate packet streams for individual sub-bands. Final ROACH design and layout was recently completed in a collaborative effort among NRAO, CASPER, and the South African KAT project. Prototype boards have been ordered, with deployment on the VLBA, the VLA, and GBT, anticipated by mid-2008.
Reaching this stage of platform completion allows NRAO engineers to turn to development of the FPGA code to implement DBE functionality. As described in greater detail in Issue 112, this effort is a collaboration with Haystack Observatory, in which NRAO is responsible for a Digital Downconverter FPGA personality that will find its principal application in narrowband spectroscopy, while Haystack will transfer the Polyphase Filterbank personality from their existing DBE-1 device to the new ROACH platform, to support primarily wideband continuum observations.
The Mark 5C wideband recording system is optimized to accept input from the DBE via an industry-standard 10G Ethernet interface, to record the packet payloads at a 4 Gbps data rate, and to deliver output to a software correlator directly over the Mark 5C unit’s bus. NRAO has joined with Haystack Observatory and Conduant Corporation, which together originated the Mark 5 recorder line, in a joint development of Mark 5C.
Final specifications for Mark 5C were adopted recently by all parties, after a preliminary review by international VLBI recording experts. These documents are available in Memos #12, #13, and #18. Mark 5C will maintain compatibility with existing Mark 5 disk modules, to preserve the investment in recording media by NRAO and other observatories.
Conduant already has in production an “Amazon” streaming interface to the disk array, which is capable of the 4 Gbps transfer rate. The only new development required for Mark 5C is a new daughter board to receive the DBE’s 10G Ethernet output; design of this board is nearing completion and was confirmed in a recent review. Prototype units have been ordered from Conduant, with delivery expected by mid-2008, and a contract is in place with Haystack for assistance in the final design, development of the control software, and testing of the initial units.
The DiFX software correlator system, and NRAO’s implementation thereof, was the subject of our latest VLBA Sensitivity Upgrade Project article, in Issue 113 of this Newsletter. Several more recent developments are particularly noteworthy.
All peripheral software required to interface DiFX to the VLBA’s end-to-end data flow is now complete. Initially, DiFX derives its control information from the same correlator job scripts used by the existing hardware system. In addition to minimizing current development effort, this approach guarantees accountable parallelism for comparison tests between the old and new systems. Eventually, however, we expect to adopt the VEX observation-description standard for this purpose. A standard FITS-IDI writer has been completed at the output from DiFX. These measures suffice for the tests that have already begun. The only remaining software required for routine use of DiFX are some operator-support interfaces.
An intermediate-scale computing cluster, consisting of twenty Intel Xeon quad-core processors, was recently purchased and installed to support the first large-scale use of DiFX. On the basis of benchmarks obtained with a more rudimentary system, described in Memo #17, NRAO estimates that the intermediate cluster can match the existing correlator’s throughput.
J. D. Romney, W. F. Brisken, S. J. Durand, G. Peck, M. D. Revnell, and R. C. Walker