Preliminary study of big data transfer over computer network

Khoruzhnikov S.E., Grudinin V.A., Sadov O.L., Shevel A.Y., Kairkanov A.B.
 pdf (261K)  / Annotation

List of references:

  1. BBCP — utility to trasnfer the data over network. — http://www.slac.stanford.edu/~abh/bbcp/.
  2. BBFTP — Utility for bulk data transfe. — http://doc.in2p3.fr/bbftp/.
  3. Big Data. — http://en.wikipedia.org/wiki/Big_data.
  4. CERN. — http://www.cern.ch/.
  5. Data Transfer Tools. — http://fasterdata.es.net/data-transfer-tools/.
  6. Facility for Antiproton and Ion Research. — http://www.fair-center.eu/.
  7. Fast Data Transfer. — http://monalisa.cern.ch/FDT/.
  8. File Transfer Service — FTS3. — http://www.eu-emi.eu/products/-/asset_publisher/1gkD/content/fts3;. — https://svnweb.cern.ch/trac/fts3.
  9. Grid. — http://toolkit.globus.org/toolkit/data/gridftp/.
  10. Gunter Dan, et al. Exploiting Network Parallelism for Improving Data Transfer Performance / High Performance Computing, Networking, Storage and Analysis (SCC). — 2012. — SC Companion. — DOI: 10.1109/SC.Companion.2012.337. — http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6496123.
  11. W. Hodson Stephen, W. Poole Stephen, M. Ruwart Thomas, W. Settlemyer Bradley. Moving Large Data Sets Over High-Performance Long Distance Networks. — One Bethel Valley Road, P.O. Box 2008 Oak Ridge: Oak Ridge National Laboratory. — 37831-6164. — / http://info.ornl.gov/sites/publications/files/Pub28508.pdf [1.12.2013] index.php?halsid=ig58511e1q1ekqq75uud43dn66&view_this_doc=hal-00825087&version=5 .
  12. Information Revolution: Big Data Has Arrived at an Almost Unimaginable Scale. — http://www.wired.com/magazine/2013/04/bigdata/.
  13. International Thermonuclear Experimental Reactor. — http://www.iter.org/.
  14. E. Johnston William, Dart Eli, Ernst Michael, Tierney Brian. Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC. — https://tnc2013.terena.org/getfile/402 (text) and https://tnc2013.terena.org/getfile/716 (presentation).
  15. R. Kaselis, S. Piperov, N. Magini, J. Flix, O. Gutsche, P. Kreuzer, M. Yang, S. Liu, N. Ratnikova, A. Sartirana, D. Bonacorsi, J. Letts. CMS Data Transfer operations after the first years of LHC collisions // Journal of Physics: Conference Series. — IOP Publishing, 2012. — V. 396. — International Conference on Computing in High Energy and Nuclear Physics 2012 (CHEP2012). — 042033.
  16. Laboratory of the Network Technology. — http://sdn.ifmo.ru/.
  17. Large Synoptic Survey Telescope. — http://www.lsst.org/lsst/.
  18. Lucinda Borovick, Richard L. Villars. White paper. The Critical Role of the Network in Big Data Applications. — http://www.cisco.com/en/US/solutions/collateral/ns340/ns517/ns224/ns944/critical_big_data_applications.pdf. — last read 1.12.2013.
  19. MutiPath TCP — Linux Kernel Implementation. — http://mptcp.info.ucl.ac.be/, http://multipathtcp.org/.
  20. Nam Hai Ah, et al. The Practical Obstacles of Data Transfer: Why researchers still love scp / Proceedings of the Third International Workshop on Network-Aware Data Management. — November 2013 NDM'13:. — http://dl.acm.org/citation.cfm?id=2534695.2534703&coll=DL&dl=ACM&CFID=563485433&C FTOKEN=25267057.
  21. Nunes Bruno, A. Astuto, Mendonca Marc, Nguyen Xuan-Nam, Obraczka Katia, Turletti Thierry. A Survey of Software-Defined Networking: Past, Present, and Future of Programmable Networks. — http://hal.inria.fr/.
  22. Open Networking Foundation White Paper Software-Defined Networking: The New Norm for Networks. — https://www.opennetworking.org/images/stories/downloads/white-papers/wp-sdnnewnorm.pdf (last read: 1.11.2013).
  23. OpenSSH. — http://openssh.org/.
  24. Patched OpenSSH. — http://sourceforge.net/projects/hpnssh/.
  25. PhEDEx — CMS Data Transfers. — https://cmsweb.cern.ch/phedex.
  26. PHEDEX data transfer system. — https://twiki.cern.ch/twiki/bin/view/CMSPublic/PhedexAdminDocsInstallation) and http://hept3.physics.umd.edu/HowToForAdmins/phedex.html.
  27. Square Kilometer Array. — http://skatelescope.org/.
  28. The Center for Large-scale Data Systems Research at the San Diego Supercomputer Center. — http://clds.sdsc.edu/ [last read 1.12.2013].
  29. The CMS Collaboration. The CMS experiment at the CERN LHC. — 2008. — JINST 3 S08004.
  30. The Rucio project is the new version of ATLAS Distributed Data Management (DDM) system services. — http://rucio.cern.ch/.
  31. Tierney Brian, Kissel Ezra, Swany Martin, Pouyoul Eric. Efficient Data Transfer Protocol for BigData. — www.es.net/assets/pubs_presos/eScience-networks.pdf.
  32. UDT: Breaking the Data Transfer Bottleneck. — http://udt.sourceforge.net/.
  33. J. Zurawski, S. Balasubramanian, A. Brown, E. Kissel, A. Lake, M. Swany, B. Tierney, M. Zekauskas. perfSONAR: On-board Diagnostics for Big Data. — http://www.es.net/assets/pubs_presos/20130910-IEEE-BigData-perfSONAR2.pdf. — last reading date: 1.11.2013.

Indexed in Scopus

Full-text version of the journal is also available on the web site of the scientific electronic library eLIBRARY.RU

The journal is included in the Russian Science Citation Index

The journal is included in the RSCI

International Interdisciplinary Conference "Mathematics. Computing. Education"

Copyright © 2009–2024 Institute of Computer Science