How to get a local big json data?
I have tried this, but I had no success:
var sa = require("./shared/resources/sa.json");
var array = new observableArrayModule.ObservableArray(sa);
Use the file-system module to read the file and then parse it with JSON.parse():
var fs = require('file-system');
var documents = fs.knownFolders.currentApp();
var jsonFile = documents.getFile('shared/resources/sa.json');
var array;
var jsonData;
jsonFile.readText()
.then(function (content) {
try {
jsonData = JSON.parse(content);
array = new observableArrayModule.ObservableArray(jsonData);
} catch (err) {
throw new Error('Could not parse JSON file');
}
}, function (error) {
throw new Error('Could not read JSON file');
});
Here's a real life example of how I'm doing it in a NativeScript app to read a 75kb/250 000 characters big JSON file.
TypeScript:
import {knownFolders} from "tns-core-modules/file-system";
export class Something {
loadFile() {
let appFolder = knownFolders.currentApp();
let cfgFile = appFolder.getFile("config/config.json");
console.log(cfgFile.readTextSync());
}
}
As of TypeScript version 2.9.x and above (in NativeScript 5.x.x is using versions 3.1.1 and above) we can now use resovleJsonModule option for tsconfig.json. With this option, the JSON files can now be imported just as modules and the code is simpler to use, read and maintain.
For example, we can do:
import config from "./config.json";
console.log(config.count); // 42
console.log(config.env); // "debug"
All we need to do is to use TypeScript 2.9.x and above and enable the propety in tsconfig.json
// tsconfig.json
{
"compilerOptions": {
"module": "commonjs",
"resolveJsonModule": true,
"esModuleInterop": true
}
}
A sample project demonstrating the above can be found here
I just wanted to add one more thing, which might be even easier. You can simply write the content of your JSON file in a data.js file, or whatever name you would like to use, and export it as an array. Then you can just require the data.js module.
Related
I have been working on a React Native project with Expo that uses a json file to store local data. I am importing the data like so
import data from '../database.json'
I am making changes (adding and removing) to the imported JSON by using data.push(new_data). These changes are not persistent when I close the app because I cannot figure out how to save them. I have looked at using the expo-file-system library as so:
import * as FileSystem from 'expo-file-system';
...
FileSystem.writeAsStringAsync(FileSystem.documentDirectory + 'database.json', data);
This is from looking at examples in the API documentations. This however always throws promise rejections and doesn't end up writing the file. Can you point me in the right direction?
Also, should I import the database.json in a different way so I will already have the uri to save it to?
The documentation doesn't give an example of it's returned props in promises, so I was overlooking it for longer than I care to admit 😅. I was really dedicated to figuring this out so I could use the Expo solution, and totally missed the return Promise for createFileAsync, so hopefully this saves someone a significant amount of time in the future.
import * as FileSystem from 'expo-file-system';
const { StorageAccessFramework } = FileSystem;
const saveFile = async () => {
const permissions = await StorageAccessFramework.requestDirectoryPermissionsAsync();
// Check if permission granted
if (permissions.granted) {
// Get the directory uri that was approved
let directoryUri = permissions.directoryUri;
let data = "Hello World";
// Create file and pass it's SAF URI
await StorageAccessFramework.createFileAsync(directoryUri, "filename", "application/json").then(async(fileUri) => {
// Save data to newly created file
await FileSystem.writeAsStringAsync(fileUri, data, { encoding: FileSystem.EncodingType.UTF8 });
})
.catch((e) => {
console.log(e);
});
} else {
alert("You must allow permission to save.")
}
}
Use AsyncStorage instead. The react native package is deprecated but working, or use #react-native-community/async-storage and convert json to string (AsyncStorage can only store strings)
Set item
import AsyncStorage from '#react-native-community/async-storage';
...
await AsyncStorage.setItem('myData', JSON.stringify(data))
Get item
const data = await AsyncStorage.getItem('myData')
I found #JayMax answer very helpful however it's only for Android.
On iOS all you need to do is use Sharing.shareAsync and then you can save data to the file. Check this example:
const fileUri = FileSystem.documentDirectory + 'data.txt';
FileSystem.writeAsStringAsync(fileUri, 'here goes your data from JSON. You can stringify it :)', {
encoding: FileSystem.EncodingType.UTF8,
});
const UTI = 'public.text';
Sharing.shareAsync(fileUri, {UTI}).catch((error) => {
console.log(error);
});
If you using AsyncStorage, it only store for small data. Maybe 6mb or 10 mb.
You can use expo fileSystem
import * as FileSystem from 'expo-file-system';
...
FileSystem.writeAsStringAsync(FileSystem.documentDirectory + 'database.json', data);
Convert your data (Type json to string) Such as this:
writeData = async () => {
var persons = ''
await axios.get(`http://192.168.0.48:4000/api/sql/student`)
.then(res => {
persons = res.data
})
await FileSystem.writeAsStringAsync(FileSystem.documentDirectory + `offline_queue_stored.json`, JSON.stringify(persons));
}
#1.If the JSON File is in your Project Folder (PC/Laptop)
import data from './database.json';
#2. If the JSON File is in your Phone
import * as FileSystem from 'expo-file-system';
import * as DocumentPicker from 'expo-document-picker';
this.state = {
fileURI: null,
};
componentDidMount = () =>{
this._pickDocument();
}
_pickDocument = async () => {
let result = await DocumentPicker.getDocumentAsync({});
this.setState({
fileURI: result.uri
})
let fileData = await FileSystem.readAsStringAsync(this.state.fileURI)
console.log(fileData)
};
From React dropzone, i receive a File object with a File.preview property whose value is a blob:url. i.e. File {preview: "blob:http://localhost:8080/52b6bad4-58f4-4ths-a2f5-4ee258ba864a"
Is there a way to convert this to json on the client? The file isnt need to be stored in a database (the convert JSON will be). I've attempted to use csvtojson but it's unable to use the file system as it uses node to power it. Ideally would like to convert this in the client if possible, once user has uploaded it. Any suggestions welcomed.
<Dropzone
name={field.name}
onDrop={(acceptedFiles, rejectedFiles) => {
acceptedFiles.forEach(file => {
console.log(file)
let tempFile = file.preview
csv()
.fromSteam(tempFile) // this errors with fs.exists not a function as its not running serverside
.on('end_parsed',(jsonArrObj)=>{
console.log(jsonArrObj)
})
})
}}
>
Yes, its possible with FileReader and csv:
import csv from 'csv';
// ...
const onDrop = onDrop = (e) => {
const reader = new FileReader();
reader.onload = () => {
csv.parse(reader.result, (err, data) => {
console.log(data);
});
};
reader.readAsBinaryString(e[0]);
}
// ...
<Dropzone name={field.name} onDrop={onDrop} />
FileReader API: https://developer.mozilla.org/en/docs/Web/API/FileReader
csv package: https://www.npmjs.com/package/csv
Inside webpack.config.js I have computed a javascript map that I'd like import as a module in the browser. I could write the data to disk and then read it back with e.g https://github.com/webpack/json-loader, but is there any more elegant ways to do this in-memory?
webpack.config:
var config = {
key: 'data'
}
// do something with webpack loaders
some.file.that.is.executed.in.browser.js:
var config = require('config')
window.alert('Config is', config.key)
webpack loaders are file preprocessor, thus there needs to be the file that you import. So create a dummy file and use json-string-loader to override the contents.
Create an empty config.json to your project.
In webpack.config.js:
var appConfig = {
key: 'data'
}
...
loaders: [
{
test: /config.json$/,
loader: 'json-string-loader?json=' + JSON.stringify(appConfig)
}
In browser:
var appConfig = require("./config.json");
// => returns {key: 'data'}
I could write the data to disk and then read it back with e.g
https://github.com/webpack/json-loader, but is there any more elegant
ways to do this in-memory?
Why? You can just do whatever you need in config.js (instead of static json) and just return a result:
var data = 'foo' + 'bar';
module.exports = {
key: data
}
I'd like to convert a CSV file to a JSON object using NodeJS. The problem is that my CSV file is hosted on a special URL.
URL : My CSV here
var fs = require("fs");
var Converter = require("csvtojson").Converter;
var fileStream = fs.createReadStream("myurl");
var converter = new Converter({constructResult:true});
converter.on("end_parsed", function (jsonObj) {
console.log(jsonObj);
});
fileStream.pipe(converter);
Issue :
Error: ENOENT, open 'C:\xampp\htdocs\GestionDettes\http:\www.banque-france.fr\fileadmin\user_upload\banque_de_france\Economie_et_Statistiques\Changes_et_Taux\page3_quot.csv'
at Error (native)
Edit #1 :
Request.get(myurl, function (error, Response, body) {
var converter = new Converter({
constructResult: true,
delimiter: ';'
});
converter.fromString(body,function(err, taux){
console.log(taux); // it works
});
});
I did just that in a module reading and writing on different protocol in different data formats. I used request to get http resources.
If you want take a look at alinex-datastore. With this module it should work like:
const DataStore = require('#alinex/datastore').default;
async function transform() {
const ds = new DataStore('http://any-server.org/my.csv');
await ds.load();
await ds.save('file:/etc/my-app.json');
}
That should do it.
I'm trying to save JSON data in my Ionic app to the local device storage. I would like to use the ngCordova File plugin. I can't seem to find any tutorials or example apps that use the exact methods they have in the docs.
Has anyone used this plugin before to save JSON data? How did you do it?
ngCordova takes away a lot of the ugliness of writing files using the file writer API.
This example has been adapted from the docs, and uses writeFile(path, file, data, replace) where the path is defined by cordova.file.DIRECTORY_TYPE, file is a string name for the file, data is the string representation of the data (so we will use JSON.stringify()). Replace is a boolean that will simply erase the existing contents of the file.
//Write using cordova.file.dataDirectory, see File System Layout section for more info
var json = {"test": "hello world"}
$cordovaFile.writeFile(cordova.file.dataDirectory, "hello.json", JSON.stringify(json), true)
.then(function (success) {
// success
}, function (error) {
// error
console.log(error); //error mappings are listed in the documentation
});
For a controller, supposing we are using controllerAs syntax it could look something like this:
angular.controller("...",['$cordovaFile' function ($cordovaFile) {
var vm = this;
vm.writeFile = function (fileName) {
ionic.Platform.ready(function(){
// will execute when device is ready, or immediately if the device is already ready.
var json = {"test": "hello world"}
$cordovaFile.writeFile(cordova.file.dataDirectory, "hello.json", JSON.stringify(json), true)
.then(function (success) {
// success
}, function (error) {
// error
console.log(error); //error mappings are listed in the documentation
});
});
};
});