I am trying to get all shared attributes (45) from server to the device (ESP32). I created a full dashboard to update/modify all of the attributes. But I am facing 2 issues :
Identify the shared attribute that device is receiving.
What if my device in not online at the time attribute is updated? I did not find a way to pull all of the attributes.
So far I receive a json with the modified attribute. But I cannot identify which of the 45 attributes the device is receiving.
Thanks a lot in advance
On the second question, a device could ask for the attributes on any of the protocols. For example over mqtt. The main idea is that you first request for the attributes to see the current values and then subscribe to see if there is any change on the attributes.
For the first question, I do not know the answer as you just get a list of key values.
finally I managed to find the solution.
Yes, one is exactly as you mention. Subscribe to get the updates.
On the first question , not knowing which is the key, and to avoid to do a long case or else if... there is a solution in ArduinoJson called JsonPair, which actually does pretty much the same. Gets the key and updates the value in the JsonDocument ( coming from a file in this case ) previously parsed. here is a link which help me with this solution , I´ve spent many many hours trying to fix the issue.
https://arduinojson.org/v6/api/jsonobject/begin_end/
Related
What's the best approach to store multiple images in database?
Upon browsing and searching for solution and approaches I found 2 options
first one is take the file name and concatenate and separate them using either comma or |
but I'm having doubts since I think that's messy and would produce errors later in the system
second is I make another table for images and add foreign key (post_id) to know which post it belongs.
Any suggestion will be appreciated.
Both methods work well. I would place the second.
It can also provide convenience when using some image upload plugins (filepond, dropzone etc.).
If I were to choose the first method, I would save the serialized arrays (with serialize()) for better bet.
Second one because it is normalized and you can have a better DX with that, No weird implementation to edit or any manipulation of post.
I have searched StackOverflow for an answer to this question, and I've been surprised to find very little information for what seems to be a very common task
Let's say I have an app that allows users to make posts. These posts can contain text, of course, but I also want the users to be able to insert images, and possibly videos.
So here's the dilemma. The first idea that comes to mind for storing these posts would be making a table like this:
CREATE TABLE posts(id INTEGER PRIMARY KEY AUTO_INCREMENT, owner VARCHAR(36) NOT NULL, message VARCHAR(MAX), _timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
id is an identifier for the post itself.
owner is an identifier for the person who created the post.
message contains the message, as text.
_timestamp represents the time created.
However, since SQL wasn't really made for storing images and other files, the images are being stored off-database. For sake of example, let's say they're stored using a product similar to Google Cloud Storage.
So, the question is, how should the message be formatted in such a way that they contain data (for example, a link) that would point to the images, without having to do too much work on the frontend code? (And without letting the user know that they're doing anything other than inserting an image).
From experience with GitHub and StackOverflow, Markdown is obviously nice, but not as user-friendly as I'd want, and doesn't work with images exactly the way I want.
I've thought about using HTML to format the message, but that brings up to main problems:
How should I store HTML in such a way that prevents XSS (Cross-site Scripting)? Should I just escape everything in such a way that it can still be read as HTML on the frontend?
Let's say this app is a mobile app. This means I would either have to make my own HTML parser or find an existing library for it.
So what is the best practice for this?
I see this type of functionality all the time, so what are those people (such as Facebook, Google, etc.) using?
Not only have I encountered this problem, but I feel like there should be a good answer for this on StackOverflow for others who encounter this problem.
Specifically, I want to know whether HTML is a good option, or if I should consider something else. As far as right now, I'm planning to use plain HTML, and make public URIs for Cloud Storage objects
Not speaking about specific implementation I would say you never want to insert the image/video data into the post.
These should always be either an attachment or a link.
So either you let the user to insert links into the post or you let them add attachments which are then uploaded to the server and link to them is placed into the post.
Let's say you have a situation where a user drops the image/video/audio/whatever data into the post. In that case you would fire an event that uploads the data to your storage and places the link into the post when it's done. That's what happens when you CTRL-C CTRL-V an image into GitHub message for example.
Regarding XSS, you should strip the inserted data off any javascript and stuff that you don't like and you should be fine. There are many libraries that can do this for you.
6.30.15 - HOW CAN I MAKE THIS QUESTION BETTER AND MORE HELPFUL TO OTHERS? FEEDBACK WOULD BE HELPFUL. THANKS!
I am developing a web application that will handle/manage a VERY LARGE data set - Currently any kind of heavy load causes the browser to lock up - whether I'm in the Django Rest Framework API or in the Dojo/Dgrid. This is kind of a dual question.
I've researched and can't find a clear way to do this on either side.
How do I limit how much the database sends at one time to the Django Rest Framework and/ or The Dojo Dgrid. The Dgrid pulls the data from the Django Rest API. The DRF pulls data directly from the MySQL database.
If I can control how much data is sent at one time, then hopefully it won't lock up the browser as much. ANY suggestions, advice, help, examples would be very helpful. Thanks in advance!
UPDATED 6.22.15 -
Alright, I FINALLY Got the pagination to work and it display the limit/offset in the headers. :) YAY!!!! I can also see the data in the Response headers. HOWEVER... the grid won't populate and I keep getting this odd error:
TypeError: transform(...) is null
return transform(value, key).toString();
instrum...tion.js (line 20)
I've gotten this error before, but I've never been able to find a solution to it. After researching, there's not much I can find on HOW to fix or really even what it is. Any help with this would be greatly appreciated!! I'm SO CLOSE to getting this thing to work correctly after WEEKS and WEEKS of beating my head against a wall. Please help! :) Thanks in advance!!!
2nd Update - This was an answer from a previous post - but I'm still not sure how to fix it. When I addressed another issue - it went away for awhile, but I still have no idea how to correct the issue.
Problem 3: "transform(...) is null return transform(value, key).toString();"
This sounds largely tangential to the original issue, but the most common cause is a widget template that is referencing a property via ${...} that doesn't actually exist in the widget.
I don't know how to answer this on the layer between DRF and the database, but as discussed in other SO questions like this one, DRF allows you to limit the amount of data sent with requests via page or offset/limit parameters.
Based on the phrasing of your question, it sounds like the client side is actually requesting too much data. I'll outline how the flow should work, so hopefully you can spot what you've missed:
A dgrid instance is set up with a collection referencing a dstore/Rest instance
The dstore/Rest instance is created with appropriate properties set. In this case, based on the DRF Documentation:
useRangeHeaders: false (this is already the default)
rangeStartParam: 'offset'
rangeCountParam: 'limit'
As a result, when the grid renders, you should see requests sent to your server e.g. endpoint?offset=0&limit=25 - if you don't see those two parameters, that would be why you're getting too much data
The server will need to query the database with the respective OFFSET and LIMIT
The server must provide a response with the expected number of items (except if it reaches the end of the data set first, which should be reflected by the total property in the response, presuming the customization in the previous SO answer I linked is used)
Ultimately, if the service is working as expected, the grid should only be requesting a handful of items at a time, and should only be firing one or two requests at any given time.
Would add as a comment, but not enough reputation at the moment ....
Your question is pretty general, but one strategy would be to allow the user to select the number of items they wish to view at a time and then allow the user to page through the data with 'next x items' and 'prev x items' buttons. Your data object query would then use the current position +/- 'x' as the index range to reduce the number of data objects sent to the browser. This is the basic flow for Ebay, Amazon, Google, or any site with thousands of items to display. The 'next' and 'prev' actions could be wired as POST requests.
Recently objects with the type 'web_link' started showing up in the items for some of the users that we work with. This currently messes with our application because we expect a 'size' field in all of the entries that Box returns and the 'web_link' type apparently doesn't have a size. I was wondering firstly why this was happening, I think it might be part of some older API that got exposed recent. I am also not sure how to replicate it since the Box API documentation doesn't mention anything about them. Right now our work around will be to just filter the response on our end, but it would be nice to let our users know how they could find and clean up these old objects if they don't need them, so is there a way to specifically search for them?
Our webapp allows users to create "weblinks" that are links to any URL they might come across on the internet.
They only show up in the folder listing API, and they are only used by a small % of users. We may remove them sometime in the near future, which is why they are not included in the documentation.
I've recently inherited a ASP.NET MVC 4 code base. One problem I noted was the use of some database ids (ints) in the urls as well in html form submissions. The code in its present state is exploitable through both URL tinkering and creating custom HTML posts with different numbers.
Now while I can easily fix the URL problems by using session state or additional auth checks i'm less sure about the database ids that get embedded into the HTML that the site spits out (i.e. I give them a drop down to fill). When the ids come back in a post how can I be sure I put them there as valid options?
What is considered "best practice" in terms of addressing this problem?
While I appreciate I could just "GUID it up" I'm hesitant to do so because I find them a pain in the ass to work with when debugging databases.
Do I have a choice here? Must I GUID to prevent easy guessing of ids or is there some kind of DRY mechanism I can use to validate the usage of ids as they come back into the site?
UPDATE: A commenter asked about the exploits I'm expecting. Lets say I spit out a HTML form with a drop down list of all the locations one can import "treasure" from. The id of the locations that the user owns are 1,2 and 3, these are provided in the HTML. But the user examines the html, fiddles with it and decides to put together a POST with the id of 4 selected. 4 is not his location, its someone else's.
Validate the ID passed against the IDs the user can modify.
It may seem tedious, but this is really the only way to make sure the user has access to what they're trying to modify. Using GUIDs without validation is security by obscurity: sure guessing them is hard, but you can potentially guess them given enough resources.
You can do this at the top of the controller before you do anything else with the posted data. If there's a violation, just throw an exception and have your global exception handler deal with it; you don't need to handle it in a pretty way since you can safely assume that the user is tampering with data in an unsupported way.
The issue you describe is known as "insecure direct object references," and the OWASP group recommends two policies for dealing with this issue:
using session-based indirect object references, and
validating all accesses to object references.
An example of Suggestion #1 would be that instead of having dropdown options 1, 2, and 3, you assign each option a GUID that is associated with the original ID in a map in the user's session. When you get a POST from that user, you check to see what object the given ID was supposed to be tied to. OWASP's ESAPI has some libraries to help with this in various languages.
But in many cases Suggestion #1 is actually counterproductive. For example, in many cases you want to have URLs that can be copy/pasted from one user to another. Process #2 is generally seen as the most foolproof way to address this issue.
You are describing Broken Access Control with Insecure Ids. Once you've identified the threat and decided which Ids are owned by certain users, ensure checks are in place for this server side.