Node.js File Upload with Multer (Complete Guide)

Node.js, Multer|SEPTEMBER 12, 2025|0 VIEWS

Master file uploads in Node.js with Multer - from basic implementation to advanced features including validation, multiple files, and cloud storage integration

Introduction

File upload functionality is a cornerstone of modern web applications. Whether you're building a social media platform that handles profile pictures, a document management system, or an e-commerce site with product images, implementing secure and efficient file uploads is crucial.

Multer is the de facto standard middleware for handling multipart/form-data in Node.js applications. It's built on top of the robust busboy library and provides a simple, yet powerful API for processing file uploads in Express.js applications.

In this comprehensive guide, we'll explore everything you need to know about implementing file uploads with Multer, from basic setup to advanced features like file validation, cloud storage integration, and security best practices.

What is Multer?

Multer is a Node.js middleware for handling multipart/form-data, which is primarily used for uploading files. It's written on top of busboy for maximum efficiency and provides a clean API that integrates seamlessly with Express.js.

Key Features

Memory and Disk Storage: Choose between storing files in memory or on disk
File Filtering: Control which files are accepted based on mimetype, size, or custom logic
Multiple Files: Handle single files, multiple files, or mixed form data
Custom Storage Engines: Integrate with cloud storage services like AWS S3
Error Handling: Comprehensive error handling for various upload scenarios
TypeScript Support: Full TypeScript definitions for type safety

Basic Setup and Installation

Let's start by setting up a basic Express.js application with Multer.

Installation

npm init -y
npm install express multer
npm install -D @types/express @types/multer @types/node nodemon typescript

Basic Express Server Setup

// server.js
const express = require("express");
const multer = require("multer");
const path = require("path");
const fs = require("fs");

const app = express();
const PORT = process.env.PORT || 3000;

// Create uploads directory if it doesn't exist
const uploadsDir = "uploads";
if (!fs.existsSync(uploadsDir)) {
  fs.mkdirSync(uploadsDir, { recursive: true });
}

// Middleware
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// Serve static files
app.use("/uploads", express.static("uploads"));

app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

Basic HTML Form

<!DOCTYPE html>
<html>
  <head>
    <title>File Upload with Multer</title>
  </head>
  <body>
    <h1>File Upload Example</h1>

    <!-- Single file upload -->
    <form action="/upload-single" method="post" enctype="multipart/form-data">
      <div>
        <label for="file">Choose file:</label>
        <input type="file" id="file" name="file" required />
      </div>
      <div>
        <button type="submit">Upload</button>
      </div>
    </form>

    <!-- Multiple file upload -->
    <form action="/upload-multiple" method="post" enctype="multipart/form-data">
      <div>
        <label for="files">Choose files:</label>
        <input type="file" id="files" name="files" multiple required />
      </div>
      <div>
        <button type="submit">Upload Files</button>
      </div>
    </form>
  </body>
</html>

Storage Configuration

Multer provides two built-in storage engines: memory storage and disk storage.

Disk Storage

Disk storage gives you full control over storing files to disk, including where and how files are named.

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    // Create directory based on file type
    const uploadPath = `uploads/${file.mimetype.split("/")[0]}s`;

    if (!fs.existsSync(uploadPath)) {
      fs.mkdirSync(uploadPath, { recursive: true });
    }

    cb(null, uploadPath);
  },
  filename: function (req, file, cb) {
    // Generate unique filename
    const uniqueSuffix = Date.now() + "-" + Math.round(Math.random() * 1e9);
    cb(
      null,
      file.fieldname + "-" + uniqueSuffix + path.extname(file.originalname)
    );
  },
});

const upload = multer({
  storage: storage,
  limits: {
    fileSize: 5 * 1024 * 1024, // 5MB limit
    files: 5, // Maximum 5 files
  },
});

Memory Storage

Memory storage stores files in memory as Buffer objects. This is useful for processing files before saving them or uploading to cloud services.

const memoryStorage = multer.memoryStorage();

const uploadToMemory = multer({
  storage: memoryStorage,
  limits: {
    fileSize: 2 * 1024 * 1024, // 2MB limit
  },
});

Custom Storage Engine

You can also create custom storage engines for specialized needs:

const customStorage = multer.memoryStorage();

function CustomStorage(options) {
  this.getDestination =
    options.destination ||
    function (req, file, cb) {
      cb(null, "/tmp/uploads");
    };
}

CustomStorage.prototype._handleFile = function (req, file, cb) {
  // Custom file handling logic
  const chunks = [];

  file.stream.on("data", (chunk) => {
    chunks.push(chunk);
  });

  file.stream.on("end", () => {
    const buffer = Buffer.concat(chunks);
    // Process the buffer as needed
    cb(null, {
      buffer: buffer,
      size: buffer.length,
    });
  });

  file.stream.on("error", cb);
};

CustomStorage.prototype._removeFile = function (req, file, cb) {
  // Cleanup logic
  cb(null);
};

File Upload Routes

Let's implement various file upload scenarios with proper error handling.

Single File Upload

// Single file upload
app.post("/upload-single", upload.single("file"), (req, res) => {
  try {
    if (!req.file) {
      return res.status(400).json({
        success: false,
        message: "No file uploaded",
      });
    }

    const fileInfo = {
      filename: req.file.filename,
      originalname: req.file.originalname,
      mimetype: req.file.mimetype,
      size: req.file.size,
      path: req.file.path,
      url: `${req.protocol}://${req.get("host")}/uploads/${req.file.filename}`,
    };

    res.json({
      success: true,
      message: "File uploaded successfully",
      file: fileInfo,
    });
  } catch (error) {
    console.error("Upload error:", error);
    res.status(500).json({
      success: false,
      message: "File upload failed",
      error: error.message,
    });
  }
});

Multiple Files Upload

// Multiple files upload (same field name)
app.post("/upload-multiple", upload.array("files", 5), (req, res) => {
  try {
    if (!req.files || req.files.length === 0) {
      return res.status(400).json({
        success: false,
        message: "No files uploaded",
      });
    }

    const filesInfo = req.files.map((file) => ({
      filename: file.filename,
      originalname: file.originalname,
      mimetype: file.mimetype,
      size: file.size,
      path: file.path,
      url: `${req.protocol}://${req.get("host")}/uploads/${file.filename}`,
    }));

    res.json({
      success: true,
      message: `${req.files.length} files uploaded successfully`,
      files: filesInfo,
    });
  } catch (error) {
    console.error("Multiple upload error:", error);
    res.status(500).json({
      success: false,
      message: "Files upload failed",
      error: error.message,
    });
  }
});

Mixed Form Data with Files

// Handle mixed form data with multiple file fields
app.post(
  "/upload-mixed",
  upload.fields([
    { name: "avatar", maxCount: 1 },
    { name: "gallery", maxCount: 8 },
    { name: "documents", maxCount: 3 },
  ]),
  (req, res) => {
    try {
      const response = {
        success: true,
        message: "Files uploaded successfully",
        data: {
          textFields: req.body,
          files: {},
        },
      };

      // Process each file field
      Object.keys(req.files).forEach((fieldName) => {
        response.data.files[fieldName] = req.files[fieldName].map((file) => ({
          filename: file.filename,
          originalname: file.originalname,
          mimetype: file.mimetype,
          size: file.size,
          url: `${req.protocol}://${req.get("host")}/uploads/${file.filename}`,
        }));
      });

      res.json(response);
    } catch (error) {
      console.error("Mixed upload error:", error);
      res.status(500).json({
        success: false,
        message: "Upload failed",
        error: error.message,
      });
    }
  }
);

File Validation and Filtering

Proper file validation is crucial for security and user experience.

File Type Filtering

// File filter function
const fileFilter = (req, file, cb) => {
  // Check file type
  const allowedTypes = /jpeg|jpg|png|gif|pdf|doc|docx/;
  const extname = allowedTypes.test(
    path.extname(file.originalname).toLowerCase()
  );
  const mimetype = allowedTypes.test(file.mimetype);

  if (mimetype && extname) {
    return cb(null, true);
  } else {
    cb(
      new Error(
        "Invalid file type. Only JPEG, PNG, GIF, PDF, DOC, and DOCX files are allowed."
      )
    );
  }
};

// Image-only filter
const imageFilter = (req, file, cb) => {
  if (file.mimetype.startsWith("image/")) {
    cb(null, true);
  } else {
    cb(new Error("Only image files are allowed!"), false);
  }
};

// Apply filter to upload configuration
const uploadWithFilter = multer({
  storage: storage,
  fileFilter: fileFilter,
  limits: {
    fileSize: 5 * 1024 * 1024, // 5MB
    files: 10,
  },
});

Advanced Validation

// Advanced file validation middleware
const validateFile = (req, res, next) => {
  if (!req.file && !req.files) {
    return res.status(400).json({
      success: false,
      message: "No file provided",
    });
  }

  const files = req.files || [req.file];
  const errors = [];

  files.forEach((file, index) => {
    // Check file size
    if (file.size > 5 * 1024 * 1024) {
      errors.push(`File ${index + 1}: Size exceeds 5MB limit`);
    }

    // Check file extension
    const allowedExtensions = [
      ".jpg",
      ".jpeg",
      ".png",
      ".gif",
      ".pdf",
      ".doc",
      ".docx",
    ];
    const fileExtension = path.extname(file.originalname).toLowerCase();

    if (!allowedExtensions.includes(fileExtension)) {
      errors.push(`File ${index + 1}: Invalid file extension`);
    }

    // Check filename for security
    const dangerousChars = /[<>:"/\\|?*\x00-\x1f]/g;
    if (dangerousChars.test(file.originalname)) {
      errors.push(`File ${index + 1}: Filename contains invalid characters`);
    }
  });

  if (errors.length > 0) {
    return res.status(400).json({
      success: false,
      message: "File validation failed",
      errors: errors,
    });
  }

  next();
};

// Use validation middleware
app.post(
  "/upload-validated",
  uploadWithFilter.single("file"),
  validateFile,
  (req, res) => {
    res.json({
      success: true,
      message: "File uploaded and validated successfully",
      file: {
        filename: req.file.filename,
        originalname: req.file.originalname,
        size: req.file.size,
        mimetype: req.file.mimetype,
      },
    });
  }
);

Error Handling

Comprehensive error handling is essential for a robust file upload system.

// Global error handler for Multer
const handleMulterError = (err, req, res, next) => {
  if (err instanceof multer.MulterError) {
    switch (err.code) {
      case "LIMIT_FILE_SIZE":
        return res.status(400).json({
          success: false,
          message: "File too large. Maximum size is 5MB.",
          error: "FILE_TOO_LARGE",
        });

      case "LIMIT_FILE_COUNT":
        return res.status(400).json({
          success: false,
          message: "Too many files. Maximum allowed is 5.",
          error: "TOO_MANY_FILES",
        });

      case "LIMIT_UNEXPECTED_FILE":
        return res.status(400).json({
          success: false,
          message: "Unexpected file field.",
          error: "UNEXPECTED_FIELD",
        });

      case "LIMIT_PART_COUNT":
        return res.status(400).json({
          success: false,
          message: "Too many parts in the form.",
          error: "TOO_MANY_PARTS",
        });

      case "LIMIT_FIELD_KEY":
        return res.status(400).json({
          success: false,
          message: "Field name too long.",
          error: "FIELD_NAME_TOO_LONG",
        });

      case "LIMIT_FIELD_VALUE":
        return res.status(400).json({
          success: false,
          message: "Field value too long.",
          error: "FIELD_VALUE_TOO_LONG",
        });

      case "LIMIT_FIELD_COUNT":
        return res.status(400).json({
          success: false,
          message: "Too many fields in the form.",
          error: "TOO_MANY_FIELDS",
        });

      default:
        return res.status(400).json({
          success: false,
          message: "File upload error",
          error: err.code,
        });
    }
  }

  // Handle custom file filter errors
  if (err.message) {
    return res.status(400).json({
      success: false,
      message: err.message,
      error: "INVALID_FILE_TYPE",
    });
  }

  next(err);
};

// Apply error handler
app.use(handleMulterError);

// General error handler
app.use((err, req, res, next) => {
  console.error("Unhandled error:", err);
  res.status(500).json({
    success: false,
    message: "Internal server error",
    error:
      process.env.NODE_ENV === "development" ? err.message : "SERVER_ERROR",
  });
});

Advanced Features

File Processing and Thumbnails

const sharp = require("sharp"); // npm install sharp

// Image processing middleware
const processImages = async (req, res, next) => {
  if (!req.file || !req.file.mimetype.startsWith("image/")) {
    return next();
  }

  try {
    const filename = req.file.filename;
    const outputPath = path.join("uploads/processed", filename);
    const thumbnailPath = path.join("uploads/thumbnails", filename);

    // Ensure directories exist
    ["uploads/processed", "uploads/thumbnails"].forEach((dir) => {
      if (!fs.existsSync(dir)) {
        fs.mkdirSync(dir, { recursive: true });
      }
    });

    // Resize and optimize main image
    await sharp(req.file.path)
      .resize(1200, 1200, {
        fit: "inside",
        withoutEnlargement: true,
      })
      .jpeg({ quality: 85 })
      .toFile(outputPath);

    // Create thumbnail
    await sharp(req.file.path)
      .resize(300, 300, {
        fit: "cover",
      })
      .jpeg({ quality: 80 })
      .toFile(thumbnailPath);

    // Add processed paths to req.file
    req.file.processedPath = outputPath;
    req.file.thumbnailPath = thumbnailPath;

    next();
  } catch (error) {
    console.error("Image processing error:", error);
    next(error);
  }
};

// Use image processing
app.post("/upload-image", upload.single("image"), processImages, (req, res) => {
  res.json({
    success: true,
    message: "Image uploaded and processed successfully",
    file: {
      original: req.file.filename,
      processed: req.file.processedPath,
      thumbnail: req.file.thumbnailPath,
    },
  });
});

Cloud Storage Integration (AWS S3)

const AWS = require("aws-sdk"); // npm install aws-sdk
const multerS3 = require("multer-s3"); // npm install multer-s3

// Configure AWS
AWS.config.update({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  region: process.env.AWS_REGION,
});

const s3 = new AWS.S3();

// S3 storage configuration
const s3Storage = multerS3({
  s3: s3,
  bucket: process.env.AWS_S3_BUCKET,
  acl: "public-read",
  metadata: function (req, file, cb) {
    cb(null, {
      fieldName: file.fieldname,
      uploadedBy: req.user?.id || "anonymous",
      uploadDate: new Date().toISOString(),
    });
  },
  key: function (req, file, cb) {
    const folder = file.mimetype.startsWith("image/") ? "images" : "documents";
    const uniqueSuffix = Date.now() + "-" + Math.round(Math.random() * 1e9);
    const filename = `${folder}/${uniqueSuffix}${path.extname(
      file.originalname
    )}`;
    cb(null, filename);
  },
});

const uploadToS3 = multer({
  storage: s3Storage,
  fileFilter: imageFilter,
  limits: {
    fileSize: 10 * 1024 * 1024, // 10MB
  },
});

// S3 upload route
app.post("/upload-s3", uploadToS3.single("file"), (req, res) => {
  if (!req.file) {
    return res.status(400).json({
      success: false,
      message: "No file uploaded",
    });
  }

  res.json({
    success: true,
    message: "File uploaded to S3 successfully",
    file: {
      filename: req.file.key,
      location: req.file.location,
      bucket: req.file.bucket,
      size: req.file.size,
      mimetype: req.file.mimetype,
    },
  });
});

Progress Tracking with WebSockets

const http = require("http");
const socketIo = require("socket.io"); // npm install socket.io

const server = http.createServer(app);
const io = socketIo(server);

// Custom storage with progress tracking
function ProgressStorage(options) {
  this.getDestination = options.destination;
}

ProgressStorage.prototype._handleFile = function (req, file, cb) {
  const uploadId = req.headers["upload-id"];
  const socket = io.sockets.sockets.get(uploadId);

  let totalSize = 0;
  let uploadedSize = 0;

  const chunks = [];

  file.stream.on("data", (chunk) => {
    chunks.push(chunk);
    uploadedSize += chunk.length;

    if (socket) {
      const progress = Math.round((uploadedSize / totalSize) * 100);
      socket.emit("upload-progress", { progress, uploadedSize, totalSize });
    }
  });

  file.stream.on("end", () => {
    const buffer = Buffer.concat(chunks);
    const filename = `${Date.now()}-${file.originalname}`;
    const filepath = path.join("uploads", filename);

    fs.writeFile(filepath, buffer, (err) => {
      if (err) return cb(err);

      if (socket) {
        socket.emit("upload-complete", { filename, size: buffer.length });
      }

      cb(null, {
        filename: filename,
        path: filepath,
        size: buffer.length,
      });
    });
  });

  file.stream.on("error", cb);
};

ProgressStorage.prototype._removeFile = function (req, file, cb) {
  fs.unlink(file.path, cb);
};

// WebSocket connection handling
io.on("connection", (socket) => {
  console.log("Client connected:", socket.id);

  socket.on("start-upload", (data) => {
    socket.emit("upload-ready", { uploadId: socket.id });
  });

  socket.on("disconnect", () => {
    console.log("Client disconnected:", socket.id);
  });
});

Security Best Practices

File Upload Security

// Security middleware
const securityMiddleware = {
  // Scan for malicious content
  scanFile: async (req, res, next) => {
    if (!req.file) return next();

    try {
      // Check file signature (magic numbers)
      const buffer = fs.readFileSync(req.file.path);
      const fileSignature = buffer.toString("hex", 0, 4);

      const allowedSignatures = {
        ffd8ffe0: "jpg",
        ffd8ffe1: "jpg",
        ffd8ffe2: "jpg",
        "89504e47": "png",
        47494638: "gif",
        25504446: "pdf",
      };

      if (!allowedSignatures[fileSignature]) {
        fs.unlinkSync(req.file.path); // Delete the file
        return res.status(400).json({
          success: false,
          message: "Invalid file signature detected",
        });
      }

      next();
    } catch (error) {
      console.error("File scan error:", error);
      next(error);
    }
  },

  // Sanitize filename
  sanitizeFilename: (req, res, next) => {
    if (!req.file) return next();

    // Remove dangerous characters and limit length
    const sanitized = req.file.originalname
      .replace(/[^a-zA-Z0-9.-]/g, "_")
      .substring(0, 100);

    req.file.originalname = sanitized;
    next();
  },

  // Rate limiting
  uploadRateLimit: require("express-rate-limit")({
    windowMs: 15 * 60 * 1000, // 15 minutes
    max: 10, // limit each IP to 10 uploads per windowMs
    message: {
      success: false,
      message: "Too many upload attempts, please try again later",
    },
  }),
};

// Apply security middleware
app.use("/upload*", securityMiddleware.uploadRateLimit);
app.post(
  "/secure-upload",
  upload.single("file"),
  securityMiddleware.sanitizeFilename,
  securityMiddleware.scanFile,
  (req, res) => {
    res.json({
      success: true,
      message: "File uploaded securely",
      file: req.file,
    });
  }
);

Input Validation and Sanitization

const validator = require("validator"); // npm install validator

// Comprehensive validation middleware
const validateUploadRequest = (req, res, next) => {
  const errors = [];

  // Validate text fields
  if (req.body.title) {
    if (!validator.isLength(req.body.title, { min: 1, max: 100 })) {
      errors.push("Title must be between 1 and 100 characters");
    }
    // Sanitize HTML
    req.body.title = validator.escape(req.body.title);
  }

  if (req.body.description) {
    if (!validator.isLength(req.body.description, { min: 0, max: 500 })) {
      errors.push("Description must be less than 500 characters");
    }
    req.body.description = validator.escape(req.body.description);
  }

  // Validate file metadata
  if (req.file) {
    const file = req.file;

    // Check for null bytes (potential security risk)
    if (file.originalname.includes("\0")) {
      errors.push("Filename contains invalid characters");
    }

    // Validate file size
    if (file.size === 0) {
      errors.push("File is empty");
    }

    // Check for executable files
    const executableExtensions = [".exe", ".bat", ".cmd", ".scr", ".pif"];
    const extension = path.extname(file.originalname).toLowerCase();
    if (executableExtensions.includes(extension)) {
      errors.push("Executable files are not allowed");
    }
  }

  if (errors.length > 0) {
    // Clean up uploaded file if validation fails
    if (req.file && req.file.path) {
      fs.unlink(req.file.path, () => {});
    }

    return res.status(400).json({
      success: false,
      message: "Validation failed",
      errors: errors,
    });
  }

  next();
};

Testing File Uploads

Unit Tests with Jest

// tests/upload.test.js
const request = require("supertest");
const path = require("path");
const fs = require("fs");
const app = require("../server");

describe("File Upload Tests", () => {
  const testImagePath = path.join(__dirname, "fixtures", "test-image.jpg");
  const testPdfPath = path.join(__dirname, "fixtures", "test-document.pdf");

  beforeAll(() => {
    // Create test fixtures
    if (!fs.existsSync(path.dirname(testImagePath))) {
      fs.mkdirSync(path.dirname(testImagePath), { recursive: true });
    }

    // Create a simple test image
    const testImageBuffer = Buffer.from([
      0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10, 0x4a, 0x46, 0x49, 0x46, 0x00, 0x01,
      0x01, 0x01, 0x00, 0x48,
    ]);
    fs.writeFileSync(testImagePath, testImageBuffer);
  });

  afterAll(() => {
    // Cleanup test files
    if (fs.existsSync(testImagePath)) {
      fs.unlinkSync(testImagePath);
    }

    // Cleanup uploaded files
    const uploadsDir = "uploads";
    if (fs.existsSync(uploadsDir)) {
      fs.readdirSync(uploadsDir).forEach((file) => {
        fs.unlinkSync(path.join(uploadsDir, file));
      });
    }
  });

  test("should upload single file successfully", async () => {
    const response = await request(app)
      .post("/upload-single")
      .attach("file", testImagePath)
      .expect(200);

    expect(response.body.success).toBe(true);
    expect(response.body.file).toHaveProperty("filename");
    expect(response.body.file).toHaveProperty("mimetype");
  });

  test("should reject file without proper field name", async () => {
    const response = await request(app)
      .post("/upload-single")
      .attach("wrongField", testImagePath)
      .expect(400);

    expect(response.body.success).toBe(false);
  });

  test("should handle multiple file upload", async () => {
    const response = await request(app)
      .post("/upload-multiple")
      .attach("files", testImagePath)
      .attach("files", testImagePath)
      .expect(200);

    expect(response.body.success).toBe(true);
    expect(response.body.files).toHaveLength(2);
  });

  test("should reject oversized files", async () => {
    // Create a large test file
    const largeBuffer = Buffer.alloc(10 * 1024 * 1024); // 10MB
    const largeFilePath = path.join(__dirname, "fixtures", "large-file.bin");
    fs.writeFileSync(largeFilePath, largeBuffer);

    const response = await request(app)
      .post("/upload-single")
      .attach("file", largeFilePath)
      .expect(400);

    expect(response.body.success).toBe(false);
    expect(response.body.error).toBe("FILE_TOO_LARGE");

    // Cleanup
    fs.unlinkSync(largeFilePath);
  });

  test("should validate file types", async () => {
    // Create a text file with image extension
    const fakeImagePath = path.join(__dirname, "fixtures", "fake.jpg");
    fs.writeFileSync(fakeImagePath, "This is not an image");

    const response = await request(app)
      .post("/secure-upload")
      .attach("file", fakeImagePath)
      .expect(400);

    expect(response.body.success).toBe(false);
    expect(response.body.message).toContain("Invalid file signature");

    // Cleanup
    fs.unlinkSync(fakeImagePath);
  });
});

Integration Tests

// tests/integration.test.js
const request = require("supertest");
const app = require("../server");
const fs = require("fs");
const path = require("path");

describe("File Upload Integration Tests", () => {
  test("complete upload workflow", async () => {
    const testData = {
      title: "Test Upload",
      description: "This is a test upload with form data",
    };

    const response = await request(app)
      .post("/upload-mixed")
      .field("title", testData.title)
      .field("description", testData.description)
      .attach("avatar", path.join(__dirname, "fixtures", "test-image.jpg"))
      .attach(
        "documents",
        path.join(__dirname, "fixtures", "test-document.pdf")
      )
      .expect(200);

    expect(response.body.success).toBe(true);
    expect(response.body.data.textFields).toMatchObject(testData);
    expect(response.body.data.files).toHaveProperty("avatar");
    expect(response.body.data.files).toHaveProperty("documents");

    // Verify files were actually saved
    const avatarFile = response.body.data.files.avatar[0];
    const documentFile = response.body.data.files.documents[0];

    expect(fs.existsSync(path.join("uploads", avatarFile.filename))).toBe(true);
    expect(fs.existsSync(path.join("uploads", documentFile.filename))).toBe(
      true
    );
  });
});

Performance Optimization

Streaming Large Files

const stream = require("stream");
const { promisify } = require("util");
const pipeline = promisify(stream.pipeline);

// Stream processing for large files
const processLargeFile = async (req, res, next) => {
  if (!req.file || req.file.size < 50 * 1024 * 1024) {
    // Skip for files < 50MB
    return next();
  }

  try {
    const readStream = fs.createReadStream(req.file.path);
    const writeStream = fs.createWriteStream(req.file.path + ".processed");

    // Process file in chunks
    const transformStream = new stream.Transform({
      transform(chunk, encoding, callback) {
        // Apply transformations (compression, encryption, etc.)
        this.push(chunk);
        callback();
      },
    });

    await pipeline(readStream, transformStream, writeStream);

    // Replace original file with processed version
    fs.renameSync(req.file.path + ".processed", req.file.path);

    next();
  } catch (error) {
    console.error("Stream processing error:", error);
    next(error);
  }
};

Memory Management

// Memory-efficient file handling
const handleLargeUpload = multer({
  storage: multer.diskStorage({
    destination: "uploads/temp",
    filename: (req, file, cb) => {
      cb(null, `temp-${Date.now()}-${file.originalname}`);
    },
  }),
  limits: {
    fileSize: 100 * 1024 * 1024, // 100MB
    files: 1,
  },
});

// Process large files without loading into memory
app.post(
  "/upload-large",
  handleLargeUpload.single("file"),
  async (req, res) => {
    try {
      const tempPath = req.file.path;
      const finalPath = path.join("uploads", req.file.filename);

      // Move file to final destination
      await fs.promises.rename(tempPath, finalPath);

      // Process file metadata without reading entire file
      const stats = await fs.promises.stat(finalPath);

      res.json({
        success: true,
        message: "Large file uploaded successfully",
        file: {
          filename: req.file.filename,
          size: stats.size,
          uploadTime: new Date().toISOString(),
        },
      });
    } catch (error) {
      console.error("Large file upload error:", error);
      res.status(500).json({
        success: false,
        message: "Large file upload failed",
      });
    }
  }
);

Conclusion

Multer provides a robust and flexible solution for handling file uploads in Node.js applications. Throughout this guide, we've covered:

Basic Setup: Getting started with Multer and Express.js
Storage Options: Memory vs. disk storage configurations
File Validation: Implementing proper file type and size validation
Error Handling: Comprehensive error management strategies
Security: Best practices for secure file uploads
Advanced Features: Image processing, cloud storage, and progress tracking
Testing: Unit and integration testing approaches
Performance: Optimization techniques for large files

Key Takeaways

Always validate files on both client and server sides
Implement proper error handling for all upload scenarios
Use appropriate storage strategies based on your needs
Prioritize security with file type validation and malware scanning
Consider performance implications for large file uploads
Test thoroughly with various file types and edge cases

Best Practices Summary

Set appropriate file size limits
Validate file types using both extension and MIME type
Sanitize file names to prevent path traversal attacks
Implement rate limiting to prevent abuse
Use cloud storage for production applications
Monitor disk space and clean up temporary files
Log all upload activities for security auditing
Consider using CDN for serving uploaded files

With these patterns and practices, you'll be able to build secure, scalable, and user-friendly file upload functionality in your Node.js applications. Remember to always stay updated with the latest security practices and regularly audit your file upload implementation.

The complete code examples from this guide are production-ready and can be adapted to fit your specific use cases. Whether you're building a simple blog with image uploads or a complex document management system, Multer provides the foundation you need to handle files efficiently and securely.

Welcome to Huy Ha's zone of coding passion