I have a PostgreSQL table that stores articles for a simple Laravel CRUD/CMS application. The PostgreSQL table also has a JsonB column to store each piece of an individual article (title, image url, body/text content, author, etc.).
So on a single article's show page I have no problem parsing the one specific JSONB column and accessing each individual piece of data for that single article, but when it comes to parsing an entire collection of articles from the same table I am wondering what the best way of doing that would be? I am having trouble doing json_decode on one specific column when using a variable to store an Eloquent query for an entire collection of articles.
I am new to parsing JSON from a data table so I am wondering if there is something built into Laravel/Eloquent that makes accessing one JSON column and parsing it easy for an entire collection from a table?
Here is basically how my PostgreSQL table looks- - only keep in mind that there are several entries not just one, and that is where I am facing the challenge of parsing an entire collection of similarly formatted Eloquent collection objects!
|-----------|-----------------------------------------|--------|----------|
| ID | JSONB | Created| Updated |
| | | (Date) | (Date) |
|-----------|-----------------------------------------|--------|----------|
| 1 | "article": { | | |
| | "title": "Here is a title", | 2/8/18 | 2/9/18 |
| | "image": "image_url.jpg", | | |
| | "body": "Lorem ipsum blah blah blah | | |
| | more long text blah blah", | | |
| | "author": "Edgar Allan Poe", | | |
| | "etc": "yeah, you get the idea..?" | | |
| | } | | |
| | | | |
|-----------|-----------------------------------------|--------|----------|
Any help offered is genuinely appreciated, even if this is a duplicated question at this point I would appreciate being pointed in a direction so I can make better progress with my code! Thank you!
My question is why would you want to store a JSON in a field if you can just create another table or create columns on your current table. This doesn't make sense to me! You are making it so hard to create a robust application if you don't follow the standard database principles.
Nevertheless you could do something like this:
$collection = (new YourEntityName())->get();
$collection->each(function ($item, $key) {
$data = json_decode($item->jsonb);
return is_null($data) []: $data;
});
This will return your json in an array. So you can access the data inside the json like:
$articles = [];
foreach($collection as $item)
{
$articles[]= $item->first()->jsonb['article'];
}
dump($articles);
Still i think this is something related to your database design but suit yourself!
Related
Community!
Story: I am trying to upload a CSV file with a huge batch of products to my e-commerce shop. But there are many very similar products, but all with every column slightly different. And luckily the plugin I use can handle this, but it needs the same title for the entire product range or some reference to its parent product. The reference is sadly not there.
Now I want to know how I can find values in a CSV file that are nearly the same (in SQL there was something called '%LIKE%') to structure the table appropriately. I can hardly describe what I want to achieve, but here is an example for what I'm looking for.
I basically want to transform this table:
+---------------+---------------+---------------+---------------+
| ID | Title | EAN | ... |
+---------------+---------------+---------------+---------------+
| 1 | AquaMat 3.6ft | 1234567890 | ... |
+---------------+---------------+---------------+---------------+
| 2 | AquaMat 3.8ft | 1234567891 | ... |
+---------------+---------------+---------------+---------------+
| 3 | AquaMat 4ft | 1234567892 | ... |
+---------------+---------------+---------------+---------------+
into this:
+---------------+---------------+---------------+---------------+
| ID | Title | EAN | ... |
+---------------+---------------+---------------+---------------+
| 1 | AquaMat | 1234567890 | ... |
+---------------+---------------+---------------+---------------+
| 2 | AquaMat | 1234567891 | ... |
+---------------+---------------+---------------+---------------+
| 3 | AquaMat | 1234567892 | ... |
+---------------+---------------+---------------+---------------+
The extra data can be scraped. Can I do this with Excel? With Macros? With Python?
Thank you for taking time and reading this.
If you have any questions, than feel free to ask.
EDIT:
The Title column contains products with completely different names and might even contain more whitespaces. And some products might have 1 attribute but others have up to 3 attributes. But this can be sorted manually.
And with nearly the same I mean as you can see in the table. The Title's are basically the same but not identical. I want to remove the attributes from them. Also, there are no other columns with any more details, only numbers and the attributes that I am trying to cut of the title!!!
Here's an idea using Python and .split():
import csv
with open('testfile.csv', 'r', encoding="utf-8-sig") as inputfile:
csv_reader = csv.reader(inputfile, delimiter=',')
with open('outputfile.csv', 'w', newline='') as outputfile:
w = csv.writer(outputfile)
header=['ID','Title','EAN','Product','Attr1','Attr2','Attr3']
w.writerow(header)
for row in csv_reader:
if row[0]=='ID':
header_row=True
pass
else:
header_row=False
list=row[1].split()
for item in list:
# if you want, you can add some other conditions on the attribute (item) in here
row.append(item)
if not header_row:
print('row: {}'.format(row))
w.writerow(row)
I think we're going to need more information about what, exactly you're trying to achieve. Is it just the extra text after the "Aquamat" (for example) that you want to remove? If so, you could simply loop through the csv file and remove anything after "Aquamat" in the "Title" column.
I assume from your description, though, that there is more to it than this.
Perhaps a starting point would be to let us know what you mean by "nearly the same". Do you want exactly what SQL means by LIKE, or something different?
EDIT:
You might check out Python's Regular Expressions: Here. If your "nearly the same" can be translated into a regex expression as described in the docs, then you could use python to loop through the csv file and search/replace terms based on the regular expression.
Are all the "nearly the same" things in the "Title" column, or could they be in other columns as well?
I have a database which contains a lot of data and although I was not involved in setting it up it is what I have to work with.
Within this database is somewhat of a lookup table. However, this table has no link to any other tables. It essentially takes the following form
ID | input | table_name |
-------------------------------------
1 | Movie | movie_tbl |
2 | Cartoon | cartoon_tbl |
3 | Animation | cartoon_tbl |
4 | Audio | audio_tbl |
5 | Picture | picture_tbl |
The table is a lot larger than the above, but the structure is as above. So what happens is someone visits my site. Here, they have an input field. Say they enter Movie then the above table is called to find the input with Movie. It then gets what table it needs to look in. I would imagine that the query would be something like
SELECT table_name FROM lookup_table WHERE input LIKE Movie;
Now that should return movie_tbl. I now know that I need to search for Movie within movie_tbl and return all the data for its row. So movie_tbl might be like this (data would be some type of data and the column names different)
ID | input | col_1 | col_2 | col_3 |
----------------------------------------------------
1 | Movie | data | data | data |
2 | Cartoon | data | data | data |
3 | Animation | data | data | data |
4 | Audio | data | data | data |
5 | Picture | data | data | data |
So now my query will be something like this
SELECT * FROM movie_tbl WHERE input LIKE Movie;
Now the tables have tens of thousands of lines of data. My real question is whether the above will be effecient or not? With the database I was given however, I do not see any other way I could do this (I cant touch the database). Is there anything I can do to make this more effecient?
Any advice appreciated
Thanks
Why are you checking for input in the 2nd table? You have already filtered the input from the first table:
SELECT table_name FROM lookup_table WHERE input LIKE Movie;
In this case you dont have to make 2 queries. Just the 2nd one should suffice. Or just having Movie data in the 2nd table and separate tables for Cartoon, Animation etc. Because then you wont be accessing the 'WHERE' clause, just:
SELECT * FROM movie_tbl;
2nd Suggestion: Use = instead of LIKE. No need for pattern matching if you know the exact input string.
I need help with a Query, i have a table like this:
| ID | codehwos |
| --- | ----------- |
| 1 | 16,17,15,26 |
| 2 | 15,32,12,23 |
| 3 | 53,15,21,26 |
I need an outpout like this:
| codehwos | number_of_this_code |
| -------- | ---------------------- |
| 15 | 3 |
| 17 | 1 |
| 26 | 2 |
I want to sum all the time a code is used in a row.
Can anyone make a query for doing it for all the code in one time?
Thanks
You have a very poor data format. You should not store lists in strings and never store lists of numbers in strings. SQL has a great data structure for storing lists. Hint: it is called a "table" not a "string".
That said, sometimes one is stuck with other people's really poor design choices. We wouldn't make them ourselves, but we still need to get something done. Assuming you have a list of codes, you can do what you want with:
select c.code, count(*)
from codes c join
table t
on find_in_set(c.code, t.codehwos) > 0
group by c.code;
If you have any influence over the data structure, then advocate for a junction table, the right way to store this data in a relational database.
How would you create a news feed such as something facebook has implemented or even LinkedIn? Is there a Generic table in the database Such as:
------------------
| Feed |
------------------
| int Id |
| string type |
| string refId |
| string text |
| Date date |
------------------
Where the refId would be the reference the the key item in another table and type would be something like Upload,Delete,Post Comment, etc and you would have to add a new Feed item to the database for whatever you want to be in the feed. Like if you uploaded multiple documents, you would do refId = "101,203,303" which references another tables keys. Any reading material which gives me an idea of this would be appreciated or even a good solution for small/medium websites.
Let's say I've got a table like this:
| RowID | LongString |
----------------------------------------
| 1 | This is a really long string |
| 2 | This is a shorter string |
How can I get a list of distinct words used in all the rows such as below:
| Result: |
-----------
| This |
| is |
| a |
| really |
| long |
| string |
| shorter |
From MySQL docs:
MySQL does not include a function to split a delimited string. Although separated data would normally be split into separate fields within a relation data, spliting such can be useful either during initial data load/validation or where such data is held in a text field.
So there is not ready-to-go solution. If I were you, I would split such string after fetching it from DB (it is easy to do in PHP, Java C# and so on).
Howewer on this site someone has wrote procedure for such task. Check it out. It is in comments section below.