A common task for any backend service is interacting with the file system. Node.js provides a powerful built-in module called fs (File System) to do just that.

Reading Files: Sync vs. Async

The fs module provides two main ways to perform operations: synchronously (blocking) and asynchronously (non-blocking).

• Synchronous (fs.readFileSync): This method blocks the event loop. The entire file is read into memory, and your code execution pauses until it's finished. This is simple but should be avoided in a server environment because it makes your application unresponsive.

JavaScript

const fs = require('fs');

try {
  // This blocks everything until the file is read.
  const data = fs.readFileSync('./my-file.txt', 'utf8');
  console.log(data);
} catch (err) {
  console.error(err);
}
console.log('This will only run AFTER the file is read.');

• Asynchronous (fs.readFile): This is the preferred method. It's non-blocking. You provide a callback function that Node will execute once the file has been read. The event loop is free to handle other tasks in the meantime.

JavaScript

const fs = require('fs');

// The callback function has two arguments: an error and the data.
fs.readFile('./my-file.txt', 'utf8', (err, data) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(data);
});
console.log('This will run immediately, BEFORE the file is read.');

Buffers: Handling Binary Data

When you read a file without specifying an encoding (like 'utf8'), Node.js gives you a Buffer. A Buffer is a chunk of memory allocated outside the V8 JavaScript engine that holds raw binary data. It's Node's way of working with data that isn't just text, like images, videos, or compressed files.

JavaScript

const fs = require('fs');

fs.readFile('./my-image.png', (err, data) => {
  if (err) throw err;
  console.log(data); // <Buffer 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 ... >
  console.log(data.length); // e.g., 5234 bytes
});

Streams: The Efficient Way to Handle Data

What happens if you try to read a 2GB file using fs.readFile? Your program will try to load the entire 2GB file into your RAM, likely crashing the process. This is where Streams come in.

A stream is a sequence of data that is moved from one point to another over time. Instead of reading the whole file at once, you read it in small, manageable chunks. This is incredibly memory-efficient.

Think of it like watching a YouTube video (streaming) versus downloading the entire movie file before you can watch it.

Here’s how you can use a readable stream to process a large file:

JavaScript

const fs = require('fs');

// Create a readable stream from a large file.
const readStream = fs.createReadStream('./large-video.mp4');

let chunkCount = 0;

// The 'data' event is emitted every time a new chunk is available.
readStream.on('data', (chunk) => {
  // Each chunk is a Buffer.
  console.log(`Received chunk ${++chunkCount} of size ${chunk.length} bytes.`);
});

// The 'end' event is emitted when there's no more data to read.
readStream.on('end', () => {
  console.log('Finished reading the file.');
});

// The 'error' event handles any errors during the read process.
readStream.on('error', (err) => {
  console.error('An error occurred:', err);
});

Using streams is the professional way to handle large I/O operations in Node.js, ensuring your application remains stable and performant.