I'm having trouble importing csv data into my nextjs app to read with d3.
I understand that static files must be in the public directory and only absolute paths will work, but although I get the absolute path with process.cwd(), it doesn't seem to find the file. I can't import the csv at the top of the js file either?
My index.js is
import { useState, useEffect } from 'react'
import path from 'path'
import Head from 'next/head'
import { csv } from "d3"
export async function getStaticProps(context) {
const dataPath = path.join(process.cwd(), 'public/data/harry_potter.csv')
console.log("DATA PATH: ", typeof dataPath)
const data = await csv(dataPath)
console.log("DATA", data)
if (!data) {
return {
notFound: true,
}
}
return {
props: {}, // will be passed to the page component as props
}
}
export default function Home(props) {
useEffect(() => {
console.log("DATA", props)
}, [])
return (
<div className="container">
<Head>
<title>d3 exercise</title>
</Head>
<main>
</main>
</div>
)
}
My path is:
C:\Users\david\Documents\d3-pluralsight\ex1\public\data\harry_potter.csv
And the nextjs app directory is
C:\Users\david\Documents\d3-pluralsight\ex1
But I still get:
TypeError: Only absolute URLs are supported
I can't import like this either
import harry from '../public/data/harry_potter.csv'
Am I missing something? Does Next not recognise csv files without extra config?
EDIT: using csv-loader loads the csv, but doesn't give access to d3's powerful type conversion functions. If I just write my own function js runs out of memory with csvs with 20000 rows
d3.csv is expecting a URL, not a file path (older versions of d3.csv would start a XHR request; newer versions retrieve via fetch). If you want to read the file off the filesystem (as opposed to serving it via HTTP), just read it into a string with something like fs and then pass the string to d3.csvParse.
Related
Looking for some help to understand what is going on here.
The Problem
We are using a translation service that requires creating JSON resource files of copy, and within these resource files, we need to add some specific keys that the service understands so it knows what should and should not be translated.
To do this as simple as possible I want to import JSON files into my code without them being tree shaken and minified. I just need the plain JSON file included in my bundle as a JSON object.
The Solution - or so I thought
The developers at the translation service have instructed me to create a webpack rule with a type of assets/source to prevent tree shaking and modification.
This almost works but the strange thing is that the JSON gets added to the bundle as a string like so
module.exports = "{\n \"sl_translate\": \"sl_all\",\n \"title\": \"Page Title\",\n \"subtitle\": \"Page Subtitle\"\n}\n";
This of course means that when I try and reference the JSON values in my JSX it fails.
Test Repo
https://github.com/lukehillonline/nextjs-json-demo
NextJs 12
Webpack 5
SSR
Steps To Reproduce
Download the test repo and install packages
Run yarn build and wait for it to complete
Open /.next/server/pages/index.js to see the SSR page
On line 62 you'll find the JSON object as a string
Open .next/static/chunks/pages/index-{HASH}.js to see the Client Side page
If you format the code you'll find the JSON object as a string on line 39
Help!
If anyone can help me understand what is going wrong or how I can improve the webpack rule to return a JSON object rather than a string that would be a massive help.
Cheers!
The Code
next.config.js
module.exports = {
trailingSlash: true,
productionBrowserSourceMaps: true,
webpack: function (config) {
config.module.rules.push({
test: /\.content.json$/,
type: "asset/source",
});
return config;
},
};
Title.content.json
{
"sl_translate": "sl_all",
"title": "Page Title",
"subtitle": "Page Subtitle"
}
Title.jsx
import content from "./Title.content.json";
export function Title() {
return <h1>{content.title}</h1>;
}
pages/index.js
import { Title } from "../components/Title/Title";
function Home({ dummytext }) {
return (
<div>
<Title />
<p>{dummytext}</p>
</div>
);
}
export const getServerSideProps = async () => {
const dummytext = "So we can activate SSR";
return {
props: {
dummytext,
},
};
};
export default Home;
I'm trying to return a json file as a controller response, but I can't get the content of the json.
import { Controller, Get, Res, HttpStatus, Query } from '#nestjs/common';
import { Response } from 'express';
import * as MOCKED_RESPONSE_TS from './data/payment-method.data'; // this ts file is imported fine
const MOCKED_RESPONSE = require('./data/payment-method-mock'); // this json file is not found
#Controller('commons')
export class CommonController {
#Get('/payment-method')
getPaymentMoethod(#Res() res: Response): any {
res.status(HttpStatus.OK).send(MOCKED_RESPONSE);
}
}
Actually the log returns: Error: Cannot find module './data/payment-method' and the app doesn't compile
I have done this with express (even with typescript) and works fine.
I don't know if i have to setup my project to read jsons (I'm newby on nest). By the moment I have created a typescript file exporting a const with the json content and I called it successfuly
I guess the problem lies in the way you import your .json file (change import instead of const)
Another advice or solution would be to leverage the .json() method of the res object (which is actually the express adapter response object).
Let's try with this code:
Your common.controller.ts file:
import { Controller, Get, Res, HttpStatus, Query } from '#nestjs/common';
import { Response } from 'express';
import * as MOCKED_RESPONSE_TS from './data/payment-method.data'; // this ts file should still be imported fine
import * as MOCKED_RESPONSE from './data/payment-method-mock.json'; // or use const inside the controller function
#Controller('commons')
export class CommonController {
#Get('/payment-method')
getPaymentMoethod(#Res() res: Response): any {
res.status(HttpStatus.OK).json(MOCKED_RESPONSE); // <= this sends response data as json
}
}
Also in your tsconfig.json file, don't forget to add this line:
tsconfig.json
{
"compilerOptions": {
// ... other options
"resolveJsonModule": true, // here is the important line, this will help VSCode to autocomplete and suggest quick-fixes
// ... other options
}
Last thoughts: you could use the sendfile() method of the res object depending on whether you want to send back a json file or the content of your json file.
Let me know if it helps ;)
first make sure you are calling it correctly.
Are you getting any response at all? if not double check your method name since its spelled like this: getPaymentMoethod and it should be this: getPaymentMethod.
Secondly I would recommend requiring outside the method and setting it to a constant.
Lastly try wrapping it in JSON.stringify() to convert the response to a json stringified object
I'm new with React and need some one with my json file Parsing problem. I am having a PerfCompare.jsx with a variable needed in the following compare. And i need this var parsing from a external JSON file(trscConfig.JSON). I am using this lines to do. but always get this SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
trscConfig.JSON
{
"server" : "http://myserver.com"
}
PerfCompare.jsx
import React, { Component } from 'react';
import { Form, Input, Button, Radio, Row, Table, Divider, Progress, Alert } from 'antd';
import math from 'mathjs';
import { stringify } from 'qs';
import PerffarmRunJSON from './lib/PerffarmRunJSON';
import JenkinsRunJSON from './lib/JenkinsRunJSON';
import benchmarkVariantsInfo from './lib/benchmarkVariantsInfo';
import './PerfCompare.css';
//import App_DATA from './trscConfig.JSON';
const server_2 = JSON.parse('./trscConfig.JSON').server;
Use fetch():
const response = await fetch('trscConfig.JSON');
const json = await response.json();
const server_2 = json.server;
Or, if your build tool doesn't support await yet:
fetch('trscConfig.JSON')
.then(response => response.json())
.then(json => {
const server_2 = json.server;
});
In either case, downloading the JSON file at runtime will mean the response will not be available immediately. If this is a React component, I suggest doing this in componentDidMount().
Alternatively, if the JSON file is a static asset:
import {server as server_2} from './trscConfig.JSON';
JSON.parse doesn't know how to make HTTP requests/read files - it just parses exactly what you've passed in. In this case, it's failing because it's trying to convert the literal string ./trscConfig.JSON into a JSON object!
There's two ways you could get this working:
Load in the JSON via your module bundler, as you're doing in the commented out line in your code. I believe Webpack (and most others) support this out of the box, but your configuration might have it disabled, intentionally or otherwise. It might also be because you're using uppercase file extensions - try it with a file that has .json in lowercase.
Use XMLHttpRequest, the Fetch API, or a third-party HTTP client library to download the JSON at runtime, and then parse the downloaded text.
I have a basic Angular web app which reads from a JSON file located on the same server as the app and parses through the JSON file in order to set certain values on objects which drive certain behavior in my app (applies css classes, etc.)
I am not able to find online and/or figure out myself how to set up the controller to read from the JSON file in a way that allows the file to be changed and Angular to dynamically reload the file once it has been changed without reloading the entire page. The JSON file is local on the server where the app is deployed, and I wanted to avoid standing up a web service just to serve a file that already exists on the same server the app is deployed.
Here is what I am doing now:
ngOnInit(): void {
// Make the HTTP request:
this.http.get('../assets/applicationLogs.json').subscribe(data => {
// Read the result field from the JSON response.
this.node_a_status= data.nodes[0].status;
this.node_b_status= data.nodes[1].status;
this.node_c_status= data.nodes[2].status;
});
}
And here is a what my JSON file looks like:
{
"nodes":[
{ "node":"Node A", "status":"processing", "errors":null },
{ "node":"Node B", "status":"processing", "errors":null },
{ "node":"Node C", "status":"inactive", "errors":null }
]
}
First, I know I will probably need to move this get logic out of ngOnInit(), but I am a little lost on how I should go about achieving the desired behavior I have described with typescript.
You're using an http request method on the file so "Poll it"... same way you would any other http JSON service. Here's a ready made poller for you to import: https://www.npmjs.com/package/rx-polling
Best thing you can do is create a service out of it and call it in ngOnInit method and use the response the same way you've shown.
import { Observable } from 'rxjs/Observable';
import 'rxjs/add/observable/dom/ajax';
import 'rxjs/add/operator/map';
import polling from 'rx-polling';
// Example of an Observable which requests some JSON data
const request$ = Observable.ajax({
url: '../assets/applicationLogs.json',
crossDomain: true
}).map(response => response.response || [])
.map(response => response.slice(0, 10)); // Take only first 10 comments
polling(request$, { interval: 5000 }).subscribe((comments) => {
console.log(comments);
}, (error) => {
// The Observable will throw if it's not able to recover after N attempts
// By default it will attempts 9 times with exponential delay between each other.
console.error(error);
});
The Angular 2 tutorials I have read place variables directly in the app.component.ts file. For example var BAR below, which pulls data though the {Foo} interface.
import {Component} from 'angular2/core';
import {Foo} from './foo';
#Component({
etc.
});
export class AppComponent {
bar = BAR;
}
var BAR: Foo[] = [
{ 'id': 1 },
{ 'id': 2 }
];
However, I have the data for BAR in a local JSON file. I don't believe {HTTP_PROVIDER} is necessary. How would I go about getting the JSON data from the external file?
Create a file with this content
export const BAR= [
{ 'id': 1 },
{ 'id': 2 }
];
save it as BarConfig.ts or some like
later use it as follows
import { BAR } from './BarConfig';
let bar= BAR;
or even better, use BAR directly where you needed
HTTP_PROVIDER is needed if you want to load a file using http.
Here is an example of how to load a local json file over http:
this.result = {friends:[]};
this.http.get('./friends.json').map((res: Response) => res.json()).subscribe(res => this.result = res);
More details here: http://www.syntaxsuccess.com/viewarticle/angular-2.0-and-http
Juste put your .json file in your static folder (/assets if you are using the angular cli) and it should work.
Best option is to create json file and store json details inside file and call that file using like below.
If you using angular-cli Keep the json file inside Assets folder (parallel to app dir) directory
return this.http.get('<json file path inside assets folder>.json'))
.map((response: Response) => {
console.log("mock data" + response.json());
return response.json();
}
)
.catch(this.handleError);
}
Note: here you only need to give path inside assets folder like assets/json/oldjson.json then you need to write path like /json/oldjson.json
If you using webpack then you need to follow above same structure inside public folder its similar like assets folder.
You can use the Http provider in angular2
Make sure you place your local json file in the www folder.
getLocalFile(){
return this.http.get('./localFileName.json').
map(res => res.json());
}
This will return you the local JSON file.