Common Data Format Archiving of Large-Scale Intelligent Transportation Systems Data for Efficient Storage, Retrieval, and Portability

Intelligent transportation system (ITS) sensor networks, such as road weather information and traffic sensor networks, typically generate enormous amounts of data. As a result, archiving, retrieval, and exchange of ITS sensor data for planning and performance analysis are becoming increasingly difficult. An efficient ITS archiving system that is compact and exchangeable and allows efficient and fast retrieval of large amounts of data is essential. A proposal is made for a system that can meet the present and future archiving needs of large-scale ITS data. This system is referred to as common data format (CDF) and was developed by the National Space Science Data Center for archiving, exchange, and management of large-scale scientific array data. CDF is an open system that is free and portable and includes self-describing data abstraction. Archiving traffic data by using CDF is demonstrated, and its archival and retrieval performance is presented for the Minnesota Department of Transportation–s 30-s traffic data collected from about 4,000 loop detectors around Twin Cities freeways. For comparison of the archiving performance, the same data were archived by using a commercially available relational database, which was evaluated for its archival and retrieval performance. This result is presented, along with reasons that CDF is a good fit for large-scale ITS data archiving, retrieval, and exchange of data.