Meteor: reading simple JSON file - json

I am trying to read a JSON file with Meteor. I've seen various answers on stackoverflow but cannot seem to get them to work. I have tried this one which basically says:
Create a file called private/test.json with the following contents:
[{"id":1,"text":"foo"},{"id":2,"text":"bar"}]
Read the file contents when the server starts (server/start.js):
Meteor.startup(function() {
console.log(JSON.parse(Assets.getText('test.json')));
});
However this seemingly very simple example does not log anything to the console. If I trye to store it in a variable instead on console.logging it and then displaying it client side I get
Uncaught ReferenceError: myjson is not defined
where myjson was the variable I stored it in. I have tried reading the JSON client side
Template.hello.events({
'click input': function () {
myjson = JSON.parse(Assets.getText("myfile.json"));
console.log("myjson")
});
}
Which results in:
Uncaught ReferenceError: Assets is not defined
If have tried all of the options described here: Importing a JSON file in Meteor with more or less the same outcome.
Hope someone can help me out

As per the docs, Assets.getText is only available on the server as it's designed to read data in the private directory, to which clients should not have access (thus the name).
If you want to deliver this information to the client, you have two options:
Use Assets.getText exactly as you have done, but inside a method on the server, and call this method from the client to return the results. This seems like the best option to me as you're rationing access to your data via the method, rather than making it completely public.
Put it in the public folder instead and use something like jQuery.getJSON() to read it. This isn't something I've ever done, so I can't provide any further advice, but it looks pretty straightforward.

The server method is OK, just remove the extra semi-colon(;). You need a little more in the client call. The JSON data comes from the callback.
Use this in your click event:
if (typeof console !== 'undefined'){
console.log("You're calling readit");
Meteor.call('readit',function(err,response){
console.log(response);
});
}
Meteor!

Related

CSV file read issue in firebase functions

I am trying to read a csv file in a firebase function so that I can process the file and do the rest operations using the data.
import * as csv from "csvtojson";
const csvFilePath = "<gdrive shared link>"
try{
console.log("First Method...")
csv()
.fromFile(csvFilePath)
.then((jsonObj: any)=>{
console.log("jsonObj....",JSON.stringify(jsonObj));
})
console.log("Second Method...")
const jsonArray=await csv().fromFile(csvFilePath);
console.log("jsonArray...", JSON.stringify(jsonArray))
}
catch(e){
console.log("error",JSON.stringify(e))
}
The above mentioned are the 2 methods I have tried for reading the csv but both shows the firebase error
'Error: File does not exist. Check to make sure the file path to your csv is correct.'
In case of 'csvFilePath' I have tried 2 methods
Just added the csv file in same folder of the function and added the code like
const csvFilePath = "./student.csv"
Added the same file to google drive and changed the access permissions to anyone with the link can read and edit and given the path to same
const csvFilePath = "<gdrive shared link>"
Both shows the same error. In case of google drive I don't want to use any sort of google credential because I was intented to read a simple csv file in firebase function.
I will start by proposing that you convert your csv to json locally or without the function and see if it works. This is because I see you are using ES6 imports which might be causing an issue since all the documentation uses require. You can also try CSV Parse or some solutions provided in this question as an alternative, trying them without the function to check if it actually works and discard it. Actually, you can upload the JSON once you have converted it from the csv, but that depends on what you are trying to do.
I think the best way to achieve this, is following the approach given in this question, that first uploads the file into cloud storage and using onFinalize() to trigger the conversion.
Also, will address these three questions that went through similar issues with the path. They were able to fix it by adding __dirname. Each one has some extra useful information.
Context for "relative paths" seems to change to the calling module if a module is imported
The csvtojson converter ignores my file name and just puts undefined
How to avoid the error which throws a csvtojson

How to benefit from tree-shaking and code-splitting while loading JSON in Nuxt?

I have a nuxt-app, where I have a lot of posts. I recently refactored the project and I won't generate all posts anymore, as it is just taking too much time to do that.
Instead I have a page where I would fetch the matching post content via url query:
www.mypage.com/posts/?post=my-post-slug
Because the content is lying in static json files, for example in:
/static/data/posts/my-post-slug.json
/static/data/posts/my-post-slug_2.json
/static/data/posts/my-post-slug_3.json
/static/data/posts/my-post-slug_n.json
I read the post https://github.com/nuxt/nuxt.js/issues/123
about how to load json in the best way.
I decided to do something like this in the fetch() hook:
// ... simplified code
async fetch() {
let postSlug = this.$route.query.post
const post = this.$axios
.get(`/posts/posts.de.${postSlug}.json`)
.then((data) => {
return data?.data
})
.catch((error) => {
console.error('error: ', error)
const code = parseInt(error.response && error.response.status)
if (code === 404) {
this.$nuxt.error({ statusCode: 404, message: 'Post not found' })
}
})
this.activePost = post?.items?.[0] || false
}
As already said, I do not generate the actual posts, but I generate all post urls in my sitemap.xml.
When running the generate in analyze mode I have now a huuuuge bundle size (app.js), and I can't understand why...
-> Check out the attached image. (Note: app.js has a ridiculous size of 34MB!!!!😕)
I don't understand why all my post jsons appear in the static and the dist part of the bundle???
I don't understand at all why they appear in there. I want them to just lie in the static folder, but not be included in the app bundle.
(you can see that there are files like events.bundle.de.json included. I need those to fetch a list of all posts for example. I do that also within my fetch hook only.
I would be very happy if somebody could point out why those files are included (twice)!
Those files are not included "twice". You need them, so you do have them locally in your static folder.
Meanwhile, you should probably put them inside of your src directory if you don't want/need to have them exposed publicly and benefit from code-splitting thanks to Webpack as explained in the post you've linked (which is still valid even if it's a 2017 one!).
Here, since you're making an axios call and using target: 'static', it will bundle your whole thing to work even without JS and it does that ahead of time. So, in order to have all the possibilities, it includes all in the final bundle I think.
If you want to only load what is needed while not shipping a big static directory, you should import them dynamically. You can use a dynamic import: load only the needed JSON by passing the actual postSlug.
PS: style-wise, prefer using async/await (.then is deprecated) and maybe also $axios.$get.
Although I think #kissu s answer is answering my question in the title, it was not the solution for my problem. For the sake of completeness I will post what I found out after long and many hours of debugging. I still don't quite understand why this even happened, but maybe someone could comment on that as well:
In my nuxt-project I am using a utility file getData.js of which I import a function getDataServer into one of my vuex store modules.
// vuex store module: store/data.js
import { getPreviewData } from '~/api/getData'
The code looks like this:
// getData.js
// Get static JSON file (e.g. basic.de.json or posts.de.1.json)
export function getDataServer(fileProps) {
return require(`~/${fileProps.path}${fileProps.name}.${fileProps.lang}${
fileProps.page ? `.${fileProps.page}` : ''
}.json`)
}
Only by importing and not even by executing that function webpack would bundle EVERY .json file it can find in my root folder into my app.js.
That is why I had a dist folder appearing in my bundle, if not deleted. (The point I talk about in my original question, where I have things included twice).
I even created additional folders and .json files to see, and they were all bundled no matter what.
Only after removing the getData.js from my project my bundle became clean.
I understand that with the require command, webpack cannot tree-shake things, so I would have expected that some code-splitting features would not work, but what I did not expect was that this bit of code would automatically get every .json in my folder...
Does anyone know why importing that function would execute it in a way that acts as a wildcard for all .jsons?
To me it still does not make any sense.
Thanks and cheers.

Exception: '<' is an invalid start of a value

I have a Blazor Webassembly project with a controller method as follows:
[HttpGet]
public async Task<List<string>> GetStatesForProfile()
{
IConfigurationSection statesSection = configuration.GetSection("SiteSettings:States");
var sections = statesSection.GetChildren();
var states = statesSection.GetChildren().Select(s => s.Key).ToList<string>();
return states;
}
The razor page calls this method:
private async Task<bool> GetStatesModel()
{
try
{
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
...
}
catch (Exception ex)
{
Console.WriteLine($"Exception: {ex.Message}, Inner: {ex.InnerException.Message}");
}
I get this Exception:
Exception: '<' is an invalid start of a value.
I read these values from appsettings.json file, And there is no '<' in values.
{
"SiteSettings": {
"States": {
"New York": ["NYC"],
"California": ["Los Angeles", "San Francisco"]
}
}
Also I put a breakpoint in the controller method and it doesn't hit.
What is this error? Is it from parsing json? and how to resolve this?
I had a very similar problem.
In the end it turned out that my browser had cached the HTML error page (I guess I had some problems with the code when I first tried it). And no matter how I tried fixing the code I still only got the error from cache. Clearing my cache also cleared the problem.
It happens when you're trying to access an API that doesn't exist. You have to check your API project connectionstring under AppSettings and make sure it's correct and running. If it's a Blazor project, you can set it as your default project, execute and see if you get a json response.
Most probably the response you are receiving is html instead of actual JSON format for the endpoint you are requesting. Please check that.
An as HTML usually starts with <html> tag, the JSON validator fails on the very first character.
You should also clear any cache, that might be interfering with the returned data. (this has helped people resolve this same issue)
I know this is an old question, but it's one of the top results when Googling the error.
I've just spent more time than I care to admit to tracking down this error. I had a straightforward Blazor hosted app, basically unchanged from the template. It worked just fine when run locally, but when published to my web host API calls failed. I finally figured out that the problem was that I was running the publish from the Client project. When I changed to the Server project it worked properly.
Hopefully my long frustration and slight stupidity will save someone else making a similar mistake.
Seems like your api is not not accessible and its returning error HTML page by default.
You can try below solution:-
I think you are using httpclient to get data to blazor application.
If you have separate projects in solution for blazor and web api,
currently your startup application may set to run blazor project only.
Change startup projects to multiple (blazor and web api app) and give httpClient url in startup of blazor application, as webApi application url, that may solve your issue.
This error indicates a mismatch of the project targeting framework version and installed runtime on the machine. So make sure that the target framework for your project matches an installed runtime - this could be verified by multiple means; one of them is to check out the Individual Components tab of the Visual Studio Installer and lookup the target version.
E.g., there is the TargetFramework attribute in the proj file:
<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>
Then launch the Visual Studio Installer, click Modify, and visit the Individual Components tab:
Install the missing runtime (.NET 5 Runtime in this case) and you're good to go.
I got the same error. Red herring. use your browser or postman to check your api endpoint is returning the json data and not some HTML. In my case my "api/companytypes" had a typo.
private CompanyType[] companytypesarray;
private List<CompanyType> CompanyTypeList;
private List<CompanyType> CompanyTypeList2;
public async Task<bool> LoadCompanyTypes()
{
//this works
CompanyTypeList = await Http.GetFromJsonAsync<List<CompanyType>>("api/companytype");
//this also works reading the json into an array first
companytypesarray = await Http.GetFromJsonAsync<CompanyType[]>("api/companytype");
CompanyTypeList2 = companytypesarray.ToList();
return true;
}
I know this is an old question, but I had the same problem. It took some searching, but I realized that the return data was in XML instead of JSON.
I'm assuming your "http" variable is of type HttpClient, so here's what I found worked for me.
By setting the "Accept" header to allow only JSON, you avoid a miscommunication between your app and the remote server.
http.DefaultRequestHeaders.Add("Accept", "application/json");
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
I had the same issue when passing in an empty string to a controller method. Creating a second controller method that doesn't accept any input variables, and just passing an empty string to the first method helped to fix my problem.
[HttpGet]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff()
{
return await GetStuff("");
}
[HttpGet("{search}")]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff(string search)
{
...
}
Versions of package
Try to update your packages to old or new version. In my case, system.net.http.json is updated from 6.0 to 5.0
Likely you are using an Asp.NetCore hosted WASM application. By default the client's App.razor has something similar to:
<CascadingAuthenticationState>
<Router AppAssembly="#typeof(Program).Assembly">
<Found Context="routeData">
<AuthorizeRouteView DefaultLayout="#typeof(MainLayout)"
RouteData="#routeData">
<NotAuthorized>
<RedirectToLogin />
</NotAuthorized>
<Authorizing>
<Loading Caption="Authorizing..."></Loading>
</Authorizing>
</AuthorizeRouteView>
</Found>
<NotFound>
<LayoutView Layout="#typeof(MainLayout)">
<p>Sorry, there's nothing at this address.</p>
</LayoutView>
</NotFound>
</Router>
</CascadingAuthenticationState>
Herein lies the problem. Since the Client and Server share the same base address, when the application cannot find "api/account/getstatesforprofile" it gives you the client's "Sorry, there's nothing at the address" page. Which is of course HTML.
I have not found the solution to this issue, but I am working on it and will reply once I find an issue.
I was having the same problem,
"JsonReaderException: '<' is an invalid start of a value."
In my case the url for the REST service was wrong.
I was using the URL from the client project. Then I looked at the Swagger screen,
https://localhost:44322/swagger/index.html
and noticed the right URL should start with "44322"...
Corrected, worked.
In my case, I had a comma (,) written mistakenly at the beginning of the appsettings.json file ...
Just check your file and verify
///////
my error details
//////
System.FormatException HResult=0x80131537 Message=Could not parse the JSON file.
Source=Microsoft.Extensions.Configuration.Json StackTrace: at line 16 This exception was originally thrown at this call stack: [External Code] Inner Exception 1: JsonReaderException: ',' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
////
For me, most of the time it is the #lauri-peltonen answer above. However, now and again, depending on who wrote the controller I have found that this will work in Swagger but not when you call it via the client (at least in this Blazor project we are on.)
[HttpGet]
[Route("prog-map-formulations")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values?formulationId=1
And I get results in Swagger but failes with the '<' OP error.
When I change ONLY the route to:
[HttpGet]
[Route("prog-map-formulations/{formulationId:int}")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values/1
And this works in both Swagger as well as from the Client side in Blazor.
Of course, once updated, I did have to clear the cache!
If you delete "obj" folder in your directory then clean the solution and rebbuild it the exception will be resolved
In all these, there is two things that was my issue and realized, first off was that Route[("api/controller")] instead of Route[("api/[controller]")], that is missing square brackets. In the second exercise I was doing, with the first experience in mind, was from the name of the database. The database had a dot in the name (Stock.Inventory). When I change the database name to StockInventory it worked. The second one I am not so sure but it worked for me.

How to load angular-formly vm.fields object from remotely-generated json?

In my application I have dynamic field sets on what is otherwise the same form. I can load them from the server as javascript includes and that works OK.
However, it would be much better to be able to load them from a separate API.
$.getJSON() provides a good way to load the json but I have not found the right place to do this. Clearly it needs to be completed before the compile step begins.
I see there is a fieldTransform facility in formly. Could this be used to transform vm.fields from an empty object to whatever comes in from the API?
If so how would I do that?
Thx. Paul
There is an example on the website that does exactly what you're asking about. It uses $timeout to simulate an async operation to load the field configuration, but you could just as easily use angular's own $http to get the json from the server. It hides the form behind an ng-if and only shows the form when the fields return (when ng-if resolves to true, it compile the template).
Thx #kent
OK, so we need to replace the getFields() promise with this
function getFields() {
return $http.get('fields-demo.json', {headers:{'Cache-Control':'no-cache'}});
}
This returns data.fields so in vm.loadingData we say
vm.fields = result[0].data;
Seems to work for OK for me.
When testing I noticed that you have to make sure there is nothing wrong with your json such as using a field type you haven't defined. In that case the resulting error message is not very clear.
Furthermore you need to deal with the situation where the source of the data is unavailable. I tried this:
function getFields() {
console.log('getting',fields_url);
return $http.get(fields_url, {headers: {'Cache-Control':'no-cache'}}).
error(function() {
alert("can't get fields from server");
//return new Promise({status:'fields server access error'}); //??
});
.. which does at least throw the alert. However, I'm not sure how to replace the promise so as to propagate the error back to the caller.
Paul

how to send really really large json object as response - node.js with express

I have been getting this error FATAL ERROR: JS Allocation failed - process out of memory and I have pinpointed it to be the problem that I am sending really really large json object to res.json (or JSON.stringify)
To give you some context, I am basically sending around 30,000 config files (each config file has around 10,000 lines) as one json object
My question is, is there a way to send such a huge json object or is there a better way to stream it (like using socket.io?)
I am using: node v0.10.33, express#4.10.2
UPDATE: Sample code
var app = express();
app.route('/events')
.get(function(req, res, next) {
var configdata = [{config:<10,000 lines of config>}, ... 10,000 configs]
res.json(configdata); // The out of memory error comes here
})
After a lot of try, I finally decided to go with socket.io to send each config file at a time rather than all config files at once. This solved the problem of out of memory which was crashing my server. thanks for all your help
Try to use streams. What you need is a readable stream that produces data on demand. I'll write simplified code here:
var Readable = require('stream').Readable;
var rs = Readable();
rs._read = function () {
// assuming 10000 lines of config fits in memory
rs.push({config:<10,000 lines of config>);
};
rs.pipe(res);
You can try increasing the memory node has available with the --max_old_space_size flag on the command line.
There may be a more elegant solution. My first reaction was to suggest using res.json() with a Buffer object rather than trying to send the entire object all in one shot, but then I realize that whatever is converting to JSON will probably want to use the entire object all at once anyway. So you will run out of memory even though you are switching to a stream. Or at least that's what I would expect.