In my code I am calling a query from my lambda function
let featured_json_data = JSON.parse(fs.readFileSync('data/jsons/featured.json'))
This works locally because my featured.json is in the directory that I am reading from. However when I deploy with serverless, the zip it generates doesn't have those files, I get a
ENOENT: no such file directory, open...
I tried packaging by adding
package:
include:
- data/jsons/featured.json
but it just doesn't work. The only way I get this to work is manually adding the json file and then change my complied handler.js code to read from the json file in the root directory.
In this screenshot I have to add the jsons then manually upload it again and in the compiled handler.js code change the directory to exclude the data/jsons
I want to actually handle this in my servereless.yml
You can load JSON files using require().
const featured_json_data = require('./featured.json')
Or better yet, convert your JSON into JS!
For working with non-JSON files, I found that process.cwd() works for me in most cases. For example:
const fs = require('fs');
const path = require('path');
export default async (event, context, callback) => {
try {
console.log('cwd path', process.cwd());
const html = fs.readFileSync(
path.resolve(process.cwd(), './html/index.html'),
'utf-8'
);
const response = {
statusCode: 200,
headers: {
'Content-Type': 'text/html'
},
body: html
};
callback(null, response);
} catch (err) {
console.log(err);
}
};
I recommend looking at copy-webpack-plugin: https://github.com/webpack-contrib/copy-webpack-plugin
You can use it to package other files to include with your Lambda deployment.
In my project, I had a bunch of files in a /templates directory. In webpack.config.js to package up these templates, for me it looks like:
const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
plugins: [
new CopyWebpackPlugin([
'./templates/*'
])
]
};
fs.readFileSync cannot find file when deploying with lambda
Check the current directory and check target directory content in deploy environment. Add appropriate code for that checking to your program/script.
Related
Using Nuxt 3, I am struggling to do something that appears simple: I would like to get a list of restaurants from an api served by nuxt/nitro in order to use it on the client side. My original file, is a .csv file stored under assets/: assets/list.csv.
Here is what I have in my component .vuefile:
//...
const { restaurants } = await useFetch('/api/restaurants')
//...
And the content of server/api/restaurants.js:
import csv from 'csvtojson'
export default defineEventHandler(async(event) => {
const data = await csv().fromFile('~/assets/list.csv')
return { data }
})
But I get an error "[500] File does not exist". I've tried many variants but always get an error here or there. Could you help me figure out the solution? Thanks.
Actually solved by realizing the following:
As the doc suggests, the assets/ directory is for assets that are processed by the bundler (Vite or Webpack). Nuxt won't serve files in the assets/ directory unless nuxt.config.ts is configured with the appropriate loader (e.g. a csv loader). Hence the 500 error.
Nuxt Content, on the other hand, is useful to automatically parse a .csv file located in the content/ directory:
In nuxt.config.ts:
modules: ["#nuxt/content"]
In the component .vue file, the following will expose the parsed csv in data.body:
const { data } = await useAsyncData("list", () => queryContent("/list").findOne())
The beauty of nuxt is that you don't need to import anything, it does it for you.
That is the flow I want:
1 - Have a normal HTML file on one folder (HTML folder)
2 - Take this file, convert is to EJS file
3 - Insert the data I want on the EJS file
4 - Convert that EJS file back to HTML file, so now I will have the "same" HTML file, but now with all the data I want.
5 - Store this HTML on AWS S3 ( Or just locally, for now )
Render an EJS file into HTML markup
Source: Render EJS File with Node.js
EJS is a templating language that uses JavaScript to generate HTML. This post will illustrate how to use Node.js with TypeScript to render an EJS file into HTML markup. Please make sure you have Node.js and npm installed first. If you are unfamiliar with Typescript please read my post describing how to compile TypeScript with npm.
EJS
Begin by creating a new EJS file named index.ejs. This file will be the template used to generate index.html. If the model is passed into the template it will render the content as a paragraph.
<!-- Sample Page -->
<h1>Sample Page</h1>
<% if (model) { %>
<%= model.content %>
<% } %>
package.json
If you don't already have a package.json created you can create one by running the command npm init and following the prompts.
You will need your package.json to include these packages:
{
"name": "package-name-goes-here",
"version": "0.0.0",
"devDependencies": {
"#types/ejs": "^2.6.2",
"#types/node": "^11.9.4",
"ejs": "^2.6.1",
"typescript": "^3.3.3333"
}
}
You can also copy the devDependencies section and run the command npm install instead of installing one at a time.
Node.js
Create a new TypeScript file named render.ts. Then add the following code to import the modules that we will use.
//imports
import util = require("util");
import fs = require("fs");
import ejs = require("ejs");
//promisify
const mkdir = util.promisify(fs.mkdir);
const readFile = util.promisify(fs.readFile);
const writeFile = util.promisify(fs.writeFile);
The first import is the util module so that we can use the promisify function. Then import the fs module for file system access. Before using three of the functions from the fs module we can promisify them allowing for the use of async/await instead of nested callbacks. The last is for EJS, and since the render file function returns a promise by default we do not need to use promisify.
Below the import statements add an async function named render. This is where the HTML output will be generated and written to a file named index.html. It needs to be marked as an async function so that the keyword await can be used. Then make sure to call the function so the code that is about to be added will execute.
async function render() {
try {
} catch (error) {
console.log(error);
}
}
render();
Before rendering our EJS file we will need a folder to put the output. So add the following to our render function:
await mkdir("dist", { recursive: true });
This will create a new directory named dist where the html output will be saved. By passing the recursive property we can ensure parent folders are created even if none are necessary. After creating the dist folder we can use EJS to render the index.ejs template to HTML. The resulting HTML string is then written to a file named index.html in the dist folder.
At this point your index.ts file should look like this:
//imports
import util = require("util");
import fs = require("fs");
import ejs = require("ejs");
//promisify
const mkdir = util.promisify(fs.mkdir);
const readFile = util.promisify(fs.readFile);
const writeFile = util.promisify(fs.writeFile);
async function render() {
try {
//create output directory
await mkdir("dist", { recursive: true });
//render ejs template to html string
const html = await ejs
.renderFile("index.ejs", { model: false })
.then((output) => output);
//create file and write html
await writeFile("dist/index.html", html, "utf8");
} catch (error) {
console.log(error);
}
}
render();
In order to run this script we need to add a tsconfig.json file to configure the TypeScript compiler. This will compile the TypeScript into JavaScript so that it can be used by node.js. Add the tsconfig file to the same folder as the render.js script.
{
"compilerOptions": {
"module": "commonjs",
"moduleResolution": "node",
"rootDir": "./",
"outDir": "./dist",
"sourceMap": true
},
"include": ["render.js"]
}
We also need to add a script to the package.json file created earlier. This script will compile render.ts and then run it using node. Your package.json should look like this:
{
"name": "package-name-goes-here",
"version": "0.0.0",
"scripts": {
"render": "tsc && node dist/render.js"
},
"devDependencies": {
"#types/ejs": "^2.6.2",
"#types/node": "^11.9.4",
"ejs": "^2.6.1",
"typescript": "^3.3.3333"
}
}
EJS render HTML
The render script can be run in a terminal window by typing the command npm run render. Make sure to run this command from the directory where your package.json is located. After running the render script you should now see a folder named dist containing a file named index.html.
The contents of index.html should look like this:
Sample Page
Notice that the conditional block containing the model content, in the index.ejs template, is not included in the html output. This is because in the render script the model was passed in as false. Now we'll create an object to pass in as the model with some sample content to the sample page.
In the render.ts file previously created, after the import statements, create an object and add a property to it called content with the value set to a sample of content.
const pageModel = {
content: "This is some sample content. Located on the sample page.",
};
Then pass this object in to the ejs.renderFile function instead of false. The render.ts file should look like this:
//imports
import util = require("util");
import fs = require("fs");
import ejs = require("ejs");
//promisify
const mkdir = util.promisify(fs.mkdir);
const readFile = util.promisify(fs.readFile);
const writeFile = util.promisify(fs.writeFile);
const pageModel = {
content: "<p>This is some sample content. Located on the sample page.</p>",
};
async function render() {
try {
//create output directory
await mkdir("dist", { recursive: true });
//render ejs template to html string
//pass pageModel in to render content
const html = await ejs
.renderFile("index.ejs", { model: pageModel })
.then((output) => output);
//create file and write html
await writeFile("dist/index.html", html, "utf8");
} catch (error) {
console.log(error);
}
}
render();
With the model object passed into the template we should now see the conditional block rendered in the index.html output file. Run the command npm run render once more.
The index.html file in the dist folder should now look like this:
<h1>Sample Page</h1>
<p>This is some sample content. Located on the sample page.</p>
The index.ejs template can now render dynamic HTML content according to the model object configured in the render.ts file and by running npm run render after each change to generate an updated index.html file.
Looking for some help to understand what is going on here.
The Problem
We are using a translation service that requires creating JSON resource files of copy, and within these resource files, we need to add some specific keys that the service understands so it knows what should and should not be translated.
To do this as simple as possible I want to import JSON files into my code without them being tree shaken and minified. I just need the plain JSON file included in my bundle as a JSON object.
The Solution - or so I thought
The developers at the translation service have instructed me to create a webpack rule with a type of assets/source to prevent tree shaking and modification.
This almost works but the strange thing is that the JSON gets added to the bundle as a string like so
module.exports = "{\n \"sl_translate\": \"sl_all\",\n \"title\": \"Page Title\",\n \"subtitle\": \"Page Subtitle\"\n}\n";
This of course means that when I try and reference the JSON values in my JSX it fails.
Test Repo
https://github.com/lukehillonline/nextjs-json-demo
NextJs 12
Webpack 5
SSR
Steps To Reproduce
Download the test repo and install packages
Run yarn build and wait for it to complete
Open /.next/server/pages/index.js to see the SSR page
On line 62 you'll find the JSON object as a string
Open .next/static/chunks/pages/index-{HASH}.js to see the Client Side page
If you format the code you'll find the JSON object as a string on line 39
Help!
If anyone can help me understand what is going wrong or how I can improve the webpack rule to return a JSON object rather than a string that would be a massive help.
Cheers!
The Code
next.config.js
module.exports = {
trailingSlash: true,
productionBrowserSourceMaps: true,
webpack: function (config) {
config.module.rules.push({
test: /\.content.json$/,
type: "asset/source",
});
return config;
},
};
Title.content.json
{
"sl_translate": "sl_all",
"title": "Page Title",
"subtitle": "Page Subtitle"
}
Title.jsx
import content from "./Title.content.json";
export function Title() {
return <h1>{content.title}</h1>;
}
pages/index.js
import { Title } from "../components/Title/Title";
function Home({ dummytext }) {
return (
<div>
<Title />
<p>{dummytext}</p>
</div>
);
}
export const getServerSideProps = async () => {
const dummytext = "So we can activate SSR";
return {
props: {
dummytext,
},
};
};
export default Home;
In my Vue project, I have mocked some data for next step development. I already save the test data in a json file. And my vue project is typical one created with Vue-Cli, and the structure for my project goes as following:
My_project
build
config
data
service_general_info.json
node_modules
src
components
component-A
component-A.vue
as you can see, all the folders are created by the vue-cli originally. And I make a new folder data and place the test data json file inside.
And I want to read in the data by axios library in an event handling function inside the component of component-A as following:
methods: {
addData() {
console.log('add json data...');
axios.get('./../../data/service_general_info.json');
},
},
I use relative path to locate the target file.But get 404 error back. So how to set the path correctly? Currently I am running the dev mode in local host.
The error message is: GET http://localhost:8080/data/service_general_info.json 404 (Not Found)
In Vue-cli project, axios can't get data from custom folder.
You should use static folder to save test json file.
So you should change axios call like this:
axios.get('/static/service_general_info.json');
This will get data from json.
If you are doing just for sake of testing then you can save it in public folder and access it directly on http root.
e.g. I have the file results.json in public folder then I can access it using http://localhost:8080/results.json
For me it didn't work using static folder. I had to put it in public folder.
I put json folder in public & then accessed it like below.
getCountries() {
return axios.get('json/country-by-abbreviation.json', { baseURL: window.location.origin })
.then((response) => { return response.data; })
.catch((error) => {
throw error.response.data;
});
}
When the http call is made from the server, axios has no idea that you're on http://localhost:8080, you have to give the full url.
Like this:
methods: {
addData() {
console.log('add json data...');
axios.get('http://localhost:8080/data/service_general_info.json');
},
},
I had this same issue, only the above solutions wouldn't work as it is being uploaded to a subdirectory. I found I needed to put it in the public/assets folder and use:
axios.get(process.env.BASE_URL+'assets/file.json')
While in vue.config.js I have set the local and live paths
module.exports = {
publicPath: process.env.NODE_ENV === 'production'
? '/path/to/app/'
: '/'
}
You can simply read a static JSON file using import. Then assign in data.
import ServiceInfo from './../../data/service_general_info.json';
export default{
data(){
return {
ServiceInfo
}
}
}
I would like to merge all json files in a directory in nodejs. The files are being uploaded by user and all i]I know is their names are device"count".json. Count being incrememented each time. I know of json-concat but how do I use it to merge all files in a directory?
jsonConcat({
src: [],
dest: "./result.json"
}, function (json) {
console.log(json);
});
You can use the fs modules to read the files in a directory and then pass them to json-concat.
const jsonConcat = require('json-concat');
const fs = require('fs');
// an array of filenames to concat
const files = [];
const theDirectory = __dirname; // or whatever directory you want to read
fs.readdirSync(theDirectory).forEach((file) => {
// you may want to filter these by extension, etc. to make sure they are JSON files
files.push(file);
})
// pass the "files" to json concat
jsonConcat({
src: files,
dest: "./result.json"
}, function (json) {
console.log(json);
});
if You read docs carefully You'll see this part:
The options object passed may have the following keys:
src:
(String) path pointing to a directory
(Array) array of paths pointing to files and/or directories
defaults to . (current working directory)
Note: if this is a path points to a single file, nothing will be done.
so here is fix:
1) move json files to some concrete path.
2) check this code:
jsonConcat({
src: './path/to/json/files',
dest: "./result.json"
}, function (json) {
console.log(json);
});
and here is the prove how it uses src param
Mostly it needs from developer not just to use 3-rd part packages, but also dive into it's sources.
in short: KISS (: