Dealing with a large amount of text data in JavaScript/VS code

Issue

I am working on a react native app that has a lot of text data and the plan is to make it work offline so its all going to be compiled in the app. Its a resource app so there are a few categories but basically it is 400 articles in 3 or 4 categories each (1200-1600 total), or at least for the higher volume ones. Currently about 30% through the first one and it is approaching 18,000 lines in an object. Would it be faster and more efficient to have each article in a json and bring it in by name compared to one object? I don't think look up would be slow in the object because there are only 400 or so ids to look up but I am running into an issue with vscode getting really slow. The text is made outside the file and I copy it in but copying the ids for reference throughout the app is getting slow and it made me reconsider how I am storing the data. I am not seeing any performance issues on the app so far as it is loaded into a flatlist but I am curious if there is a better way to handle it in code.

I looked around online and saw more info on rendering it, which is not an issue and I think this is a bit more of a VSCode issue.

Solution

If your articles are primarily intended for reading, consider organizing them efficiently. You can create a folder structure that categorizes articles into separate folders. In React Native, you can then import articles using a clear folder structure like this:

import article1 from '../articles/category1/article1.json';

This approach simplifies article management and enhances overall organization.

Making dynamic: Assuming you have the same project structure

project/
├─ src/
│   ├─ articles/
│   │   ├─ category1/
│   │   │   ├─ article1.json
│   │   │   ├─ article2.json
│   │   │   └─ ...
│   │   ├─ category2/
│   │   │   ├─ article1.json
│   │   │   ├─ article2.json
│   │   │   └─ ...
│   │   ├─ category3/
│   │   │   ├─ article1.json
│   │   │   ├─ article2.json
│   │   │   └─ ...

You can structure your code as follows for dynamical import article:

import React, { useEffect } from 'react';

async function loadArticle(category, articleName) {
  try {
    const articleModule = require(`../articles/${category}/${articleName}.json`);
    return articleModule.default; // Assuming articles are exported as default
  } catch (error) {
    console.error(`Error loading article: ${error}`);
    return null;
  }
}

const YourComponent = () => {
  const category = 'category1';
  const articleName = 'article1';

  useEffect(() => {
    async function fetchArticle() {
      const article = await loadArticle(category, articleName);
      if (article) {
        console.log(article);
      } else {
        console.log('Article not found or error loading.');
      }
    }
    fetchArticle();
  }, [category, articleName]);

  return (
    // Your component JSX here
  );
};

export default YourComponent;

Answered By - talhamaqsood890