c++ - read input files, fastest way possible?

Question

I have numerous text files of data in the form of float numbers. I'm looking for the fastest way to read them in C++. I can change the file to binary if that's the fastest.

It would be great if you could give me hint or refer me to a website with complete explanation. I don't know whether there is any library that does the work fast. Even if there is any open source software that does the work, that would be helpful.

Answer 1

Having a binary file is the fastest option. Not only you can read it directly in an array with a raw istream::read in a single operation (which is very fast), but you can even map the file in memory if your OS supports it; you can use open/mmap on POSIX systems, CreateFile/CreateFileMapping/MapViewOfFile on Windows, or even the Boost cross-platform solution (thanks @Cory Nelson for pointing it out).

Quick & dirty examples, assuming the file contains the raw representation of some floats:

"Normal" read:

#include <fstream>
#include <vector>

// ...

// Open the stream
std::ifstream is("input.dat");
// Determine the file length
is.seekg(0, std::ios_base::end);
std::size_t size=is.tellg();
is.seekg(0, std::ios_base::beg);
// Create a vector to store the data
std::vector<float> v(size/sizeof(float));
// Load the data
is.read((char*) &v[0], size);
// Close the file
is.close();

Using shared memory:

#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/mapped_region.hpp>

using boost::interprocess;

// ....

// Create the file mapping
file_mapping fm("input.dat", read_only);
// Map the file in memory
mapped_region region(fm, read_only);
// Get the address where the file has been mapped
float * addr = (float *)region.get_address();
std::size_t elements  = region.get_size()/sizeof(float);