How do I concatenate json zip files in Python? - json

I have a file with, containing 61 zipfiles which each contain 17280 zip files.
The file exists of sensordata for the months April and May and the 61 zipfiles exist of the sensordata per day for these months. The 17280 zip files contain the data for these days per 5 seconds.
I wrote a code to open the data per day in Jupyter Notebook, but I want to open them per month. Here is my code for opening the data per day:
path = 'test_tudelft'
data = pd.DataFrame() # verzameling van alle data
for zip_filename in os.listdir(path): # loop over de zipbestanden
with zipfile.ZipFile(os.path.join(path, zip_filename)) as zf: # open een zipbestand
for file in zf.filelist: # loop over de ingepakte bestanden
# data = pd.concat((data, pd.read_json(zf.open(file), lines=True))) # lees de nieuwe data en voeg samen
data_new = pd.read_json(zf.read(file).decode('utf8')[2:-1], orient='index').T
data = pd.concat((data, data_new)) # lees de nieuwe data en voeg samen
data = data.reset_index(drop=True) # unieke index per bestand
This code works to make plots of the data per day, but I would like to plot the data per month. How can I change my code to open the zipfiles in the zipfiles?

Related

List data within bracket - Freemarker

I am new to freemarker and trying to extract list of value defined within bracket. Example below..
alternative-id provider="group" level="episode" description="CHICAGO MED YR 5 (Sign) Ep 514">20595987</alternative-id
alternative-id provider="group" level="episode" description="CHICAGO MED YR 5 (U) Ep 514">20670620</alternative-id
Would like to list values within the ()
Output will be as below
Sign
U
OR
Sign,U
Couldn't get list command working to extract data.. it only extract data within ><
20595987
20670620
Tested this successfully on https://try.freemarker.apache.org/:
Template
<#assign bracketValues = data?matches("<alternative-id([\\w\\s\=\"]*)\\(([\\w\\s\\d]*)\\)")>
<#list bracketValues as b>
${b?groups[2]}<#sep>,</#list>
Data model
data = "<alternative-id provider=\"group\" level=\"episode\" description=\"CHICAGO MED YR 5 (Sign) Ep 514\">20595987</alternative-id><alternative-id provider=\"group\" level=\"episode\" description=\"CHICAGO MED YR 5 (U) Ep 514\">20670620</alternative-id>"
Result
Sign,U

Import XML to MYSQL

I need to create Mysql tables using this XML but i can't find the way to import all the child's data.
This is the XML.
<product>
<ref>2101</ref>
<name>VERSALLES</name>
<printjobs>
<printjob>
<teccode>100115</teccode>
<tecname>TAMPOGRAFÍA E
</tecname>
<areas>
<area>
<areacode>384</areacode>
<areaname>En la tapa</areaname>
</area>
</areas>
</printjob>
<printjob>
<teccode>101200</teccode>
<tecname>TERMOGRABADO SECO
</tecname>
<areas>
<area>
<areacode>488</areacode>
<areaname>Esquina inferior derecha de la portada</areaname>
</area>
</areas>
</printjob>
<printjob>
<teccode>101201</teccode>
<tecname>TERMOGRABADO CINTA COLOR
</tecname>
<areas>
<area>
<areacode>488</areacode>
<areaname>Esquina inferior derecha de la portada</areaname>
</area>
</areas>
</printjob>
</printjobs>
</product>
<product>
<ref>2233</ref>
<name>1L</name>
<printjobs>
<printjob>
<teccode>100115</teccode>
<tecname>TAMPOGRAFÍA E
</tecname>
<areas>
<area>
<areacode>259</areacode>
<areaname>En el cuerpo</areaname>
</area>
</areas>
</printjob>
</printjobs>
</product>
I tried to import using LOAD DATA LOCAL INFILE to create a different table for area, printjob and product but its only imports the first area of every product.
This is the SQL that i used for area.
LOAD DATA LOCAL INFILE '/var/lib/mysql/allprintdatafile_esp.xml'
INTO TABLE areas
CHARACTER SET 'utf8'
LINES STARTING BY '<product>' TERMINATED BY '</product>'
(#product)
SET
id = NULL,
id_area = ExtractValue(#product, '//areacode'),
id_print_job = ExtractValue(#product, '//teccode'),
id_product = ExtractValue(#product, '//ref'),
areaname = ExtractValue(#product, '//areaname'),
);
Once imported I need the table to look like this.
id
id_area
id_print_job
id_product
areaname
1
384
100115
2101
En la tapa
2
488
101200
2101
Esquina inferior derecha de la portada
3
488
101201
2101
En la tapa
4
259
100115
2233
En el cuerpo
Thank you.

Is there a way to extract part of a dataframe using pandas (or other library) in Python?

Hi I am new on Python and pandas. So please excuse my mistakes.
I want to know if it's possible to use a dataframe and sort it with pandas.
I'm still in class, but I'd like to advance in my projects.
I want / I'd like to extract part of the JSON file.
I want all the system_name =Métro de Paris to be in a JSON file.
Following are the current code I have (JSON and script).
Thanks in advance for your help!
import pandas as pd
# parse according to "system_name":"Métro de Paris"
data = pd.read_json(r'paris_lines_systems_and_modes.json')
[{"id":406,"name":"Ancien M 2nord","url_name":"406-ancien-m-2nord","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":405,"name":"Ancien M 2sud","url_name":"405-ancien-m-2sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":399,"name":"Ancien M 14","url_name":"399-ancien-m-14","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":460,"name":"Ancienne ligne A Nord-Sud","url_name":"460-ancienne-ligne-a-nord-sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":459,"name":"Ancienne ligne B Nord-Sud","url_name":"459-ancienne-ligne-b-nord-sud","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":411,"name":"Austerlitz-Auteuil","url_name":"411-austerlitz-auteuil","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":471,"name":"CDG Express","url_name":"471-cdgexpress","color":"#f0871f","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":465,"name":"CDGVal 1","url_name":"465-cdgval-1","color":"#0a3071","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":466,"name":"CDGVal 2","url_name":"466-cdgval-2","color":"#0a3071","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":410,"name":"Charenton-Auteuil","url_name":"410-charenton-auteuil","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":407,"name":"Funiculaire de Monmartre","url_name":"407-funiculaire","color":"#601e92","system_id":302,"system_name":"Funiculaire","transport_mode_id":0,"transport_mode_name":"default"},{"id":381,"name":"H","url_name":"381-h","color":"#7b4339","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":382,"name":"J","url_name":"382-j","color":"#cdcd00","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":383,"name":"K","url_name":"383-k","color":"#c7b300","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":384,"name":"L","url_name":"384-l","color":"#7584bc","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":472,"name":"Le Bus Direct 1","url_name":"472-le-bus-direct-1","color":"#b20066","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":473,"name":"Le Bus Direct 2","url_name":"473-le-bus-direct-2","color":"#7bbb58","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":474,"name":"Le Bus Direct 3","url_name":"474-le-bus-direct-3","color":"#0098d8","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":475,"name":"Le Bus Direct 4","url_name":"475-le-bus-direct-4","color":"#ff9b00","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":8,"transport_mode_name":"bus"},{"id":412,"name":" Le Louvre-Ablon","url_name":"412-le-louvre-ablon","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":643,"name":"Ligne Horizon","url_name":"643-ligne-horizon","color":"#0092e2","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":352,"name":"M 1","url_name":"352-ligne-1","color":"#ffcd00","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":353,"name":"M 2","url_name":"353-ligne-2","color":"#006db8","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":354,"name":"M 3","url_name":"354-ligne-3","color":"#9b993b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":355,"name":"M 3bis","url_name":"355-ligne-3bis","color":"#88d3df","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":356,"name":"M 4","url_name":"356-ligne-4","color":"#ba4a9a","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":357,"name":"M 5","url_name":"357-ligne-5","color":"#f78e4b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":358,"name":"M 6","url_name":"358-ligne-6","color":"#77c596","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":303,"name":"M 7","url_name":"303-ligne-7","color":"#f49eb2","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":359,"name":"M 7bis","url_name":"359-ligne-7bis","color":"#77c596","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":360,"name":"M 8","url_name":"360-ligne-8","color":"#c5a3cc","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":361,"name":"M 9","url_name":"361-ligne-9","color":"#cdc92b","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":362,"name":"M 10","url_name":"362-ligne-10","color":"#dfb03a","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":363,"name":"M 11","url_name":"363-ligne-11","color":"#8c6539","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":364,"name":"M 12","url_name":"364-ligne-12","color":"#008c59","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":365,"name":"M 13","url_name":"365-ligne-13","color":"#87d2df","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":366,"name":"M 14","url_name":"366-ligne-14","color":"#662c91","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":395,"name":"M 15","url_name":"395-m-15","color":"#a60f31","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":396,"name":"M 16","url_name":"396-m-16","color":"#ec7cae","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":397,"name":"M 17","url_name":"397-m-17","color":"#ec7cae","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":398,"name":"M 18","url_name":"398-m-18","color":"#95bf32","system_id":300,"system_name":"Grand Paris Express","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":385,"name":"N","url_name":"385-n","color":"#00a092","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":476,"name":"Navette","url_name":"476-navette","color":"#000","system_id":83,"system_name":"Métro de Paris","transport_mode_id":4,"transport_mode_name":"heavy_rail"},{"id":394,"name":"Orlyval","url_name":"394-orlyval","color":"#7d7e7e","system_id":299,"system_name":"Dessertes aéroportuaires","transport_mode_id":7,"transport_mode_name":"people_mover"},{"id":386,"name":"P","url_name":"386-p","color":"#f0b600","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":387,"name":"R","url_name":"387-r","color":"#e4b4d1","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":376,"name":"RER A","url_name":"376-rer-a","color":"#ec1b2a","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":377,"name":"RER B","url_name":"377-rer-b","color":"#4c90cc","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":378,"name":"RER C","url_name":"378-rer-c","color":"#ffdc1d","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":379,"name":"RER D","url_name":"379-rer-d","color":"#00ab66","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":380,"name":"RER E","url_name":"380-rer-e","color":"#c67db4","system_id":297,"system_name":"RER","transport_mode_id":3,"transport_mode_name":"commuter_rail"},{"id":414,"name":"STCRP","url_name":"414-stcrp","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":367,"name":"T 1","url_name":"367-ligne-1","color":"#14609e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":368,"name":"T 2","url_name":"368-ligne-2","color":"#ba498d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":369,"name":"T 3a","url_name":"369-ligne-3a","color":"#ef7d2d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":370,"name":"T 3b","url_name":"370-ligne-3b","color":"#009b41","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":371,"name":"T 4","url_name":"371-ligne-4","color":"#d9922b","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":372,"name":"T 5","url_name":"372-ligne-5","color":"#914e8e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":373,"name":"T 6","url_name":"373-ligne-7","color":"#e95c36","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":375,"name":"T 7","url_name":"375-t-7","color":"#885f30","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":374,"name":"T 8","url_name":"374-t-8","color":"#8c8235","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":389,"name":"T 9","url_name":"389-t-9","color":"#009cb4","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":390,"name":"T 10","url_name":"390-t-10","color":"#8c8235","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":391,"name":"T 11","url_name":"391-t-11","color":"#df5f3d","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":506,"name":"T 11 express","url_name":"506-t-11-express","color":"#e2562e","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":392,"name":"T 12","url_name":"392-t-12","color":"#a02a3a","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":393,"name":"T 13","url_name":"393-t-13","color":"#885f30","system_id":296,"system_name":"Tramway d'Île-de-France","transport_mode_id":5,"transport_mode_name":"light_rail"},{"id":400,"name":"T Zen 1","url_name":"400-t-zen-1","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":401,"name":"T Zen 2","url_name":"401-t-zen-2","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":402,"name":"T Zen 3","url_name":"402-t-zen-3","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":403,"name":"T Zen 4","url_name":"403-t-zen-4","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":404,"name":"T Zen 5","url_name":"404-t-zen-5","color":"#c6cd01","system_id":301,"system_name":"TZen","transport_mode_id":6,"transport_mode_name":"brt"},{"id":409,"name":"Tuileries-Suresnes","url_name":"409-tuileries-suresnes","color":"#000","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"},{"id":388,"name":"U","url_name":"388-u","color":"#d60058","system_id":298,"system_name":"Transilien","transport_mode_id":2,"transport_mode_name":"inter_city_rail"},{"id":408,"name":"Voguéo","url_name":"408-vogueo","color":"#99d420","system_id":303,"system_name":"Navettes fluviales","transport_mode_id":0,"transport_mode_name":"default"}]
So I managed to find how do extract part of the data you want to use / analyse.
You should simply call the column you are trying to extract (in my case system_name) and give its value (in my case "Métro de Paris"). Here The dataframe, data, will now have the data with value "Métro de Paris" form the column "system_name".
data = data[data.system_name == "Métro de Paris"]

Connection refused by google sheets API, because daily quota reached

gspread.exceptions.APIError: {'code': 429, 'message': "Quota exceeded for quota group 'ReadGroup' and limit 'Read requests per user per 100 seconds' of service 'sheets.googleapis.com' for consumer 'project_number:changedthis'.", 'status': 'RESOURCE_EXHAUSTED', 'details': [{'#type': 'type.googleapis.com/google.rpc.Help', 'links': [{'description': 'Google developer console API key', 'url': 'https://console.developers.google.com/project/changedthis/apiui/credential'}]}]}
My code sends data every 60 seconds and this only filling in 3 cells. Now I get the error at the start of my program. Do I wait for a few days or is this a permanent block?
This is related to the usage limit
As per the Sheets Documentation says There is no daily usage limit but limited requests per 100 seconds. If you are owner of a billing account you can increase this quota. Otherwise 500 requests per 100 seconds per project and 100 requests per 100 seconds per user are the limit.
As a workaround make use of different service accounts in order to reach 500 requests / 100 seconds.
Edit
Looking at your code these are the changes that are currently working (Take a look into the scopes, some of them are not necessary in this case, although I didn't modify them):
import gspread
import time
import datetime
from google.oauth2 import service_account
SCOPES = ["https://spreadsheets.google.com/feeds", 'https://www.googleapis.com/auth/spreadsheets',
"https://www.googleapis.com/auth/drive.file", "https://www.googleapis.com/auth/drive"]
SERVICE_ACCOUNT_FILE = 'creds.json'
#creds = ServiceAccountCredentials.from_json_keyfile_name("creds.json", scope)
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
creds = credentials.with_subject('user#yourdomain.com')
client = gspread.authorize(creds)
sheet = client.open_by_key("ID").sheet1 # Open the spreadsheet
i = 1
while True:
cell = sheet.cell(i, 1).value
i += 1
if len(cell) == 0: # of een manier om te kijken of de cell leeg is
print("Empty row has been found at row", (i-1))
break
# ofzo, als er geen begrenzing is en geen lege cell dan blijft deze lus maar doorgaan totdat sheet.cell(i, 1).value fout gaat denk ik
if i > 1000:
print("Did not find any empty rows")
break
while True:
x = str(datetime.datetime.now())
sheet.update_cell(i, 1, x) # Update one cell
time.sleep(60)
In order to make this work, you should activate the domain-wide authority
References
Sheets API Usage Limits
Resolve a 429 error: Too many requests

issue parsing Twitter JSON in Google App Script

I'm using the code described in the GAS documentation to retrieve my Twitter timeline.
The lines
var response = twitterService.fetch('https://api.twitter.com/1.1/statuses/home_timeline.json?count=1');
response=response.getContentText();
yield the following result:
[{"created_at":"Sun Jan 14 15:30:00 +0000 2018","id":952563458616172544,"id_str":"952563458616172544","text":"#UltimOra, \u201cAi piedi le Clarks\u201d; #LiberieUguali trova il primo punto di convergenza fra i vari leader https:\/\/omissis","truncated":false,"entities":{"hashtags":[{"text":"UltimOra","indices":[0,9]},{"text":"LiberieUguali","indices":[33,47]}],"symbols":[],"user_mentions":[],"urls":[{"url":"https:\/\/omissis","expanded_url":"http:\/\/www.lercio.it\/ai-piedi-le-clarks-liberi-e-uguali-trova-il-primo-punto-di-convergenza-fra-i-vari-leader\/","display_url":"lercio.it\/ai-piedi-le-cl\u2026","indices":[102,125]}]},"source":"\u003ca href=\"https:\/\/about.twitter.com\/products\/tweetdeck\" rel=\"nofollow\"\u003eTweetDeck\u003c\/a\u003e","in_reply_to_status_id":null,"in_reply_to_status_id_str":null,"in_reply_to_user_id":null,"in_reply_to_user_id_str":null,"in_reply_to_screen_name":null,"user":{"id":910827588,"id_str":"910827588","name":"lercio-notizie","screen_name":"lercionotizie","location":"","description":"Dal 13 aprile 2017 in fumetteria e libreria con \"Lercio, lo sporco che fa notizia. Il libro\" (Shockdom)","url":"http:\/\/omissis","entities":{"url":{"urls":[{"url":"http:\/\/omissis","expanded_url":"http:\/\/www.lercio.it","display_url":"lercio.it","indices":[0,22]}]},"description":{"urls":[]}},"protected":false,"followers_count":491614,"friends_count":25,"listed_count":712,"created_at":"Sun Oct 28 18:19:33 +0000 2012","favourites_count":385,"utc_offset":3600,"time_zone":"Amsterdam","geo_enabled":false,"verified":false,"statuses_count":7335,"lang":"it","contributors_enabled":false,"is_translator":false,"is_translation_enabled":false,"profile_background_color":"FFFFFF","profile_background_image_url":"http:\/\/pbs.twimg.com\/profile_background_images\/696203701\/001bc779080491195bc16054af629016.png","profile_background_image_url_https":"https:\/\/pbs.twimg.com\/profile_background_images\/696203701\/001bc779080491195bc16054af629016.png","profile_background_tile":false,"profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/2932719853\/4bcd23b39293870bf1832aec72b6fe3d_normal.png","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/2932719853\/4bcd23b39293870bf1832aec72b6fe3d_normal.png","profile_banner_url":"https:\/\/pbs.twimg.com\/profile_banners\/910827588\/1467828171","profile_link_color":"9B1212","profile_sidebar_border_color":"000000","profile_sidebar_fill_color":"DDEEF6","profile_text_color":"333333","profile_use_background_image":true,"has_extended_profile":false,"default_profile":false,"default_profile_image":false,"following":true,"follow_request_sent":false,"notifications":false,"translator_type":"none"},"geo":null,"coordinates":null,"place":null,"contributors":null,"is_quote_status":false,"retweet_count":73,"favorite_count":186,"favorited":false,"retweeted":false,"possibly_sensitive":false,"possibly_sensitive_appealable":false,"lang":"it"}]
But when I add the code line
response=JSON.parse(response);
the JSON file is totally altered, it doesn't even look a JSON file and its content becomes:
{possibly_sensitive_appealable=false, in_reply_to_status_id_str=null, in_reply_to_status_id=null, created_at=Sun Jan 14 15:30:00 +0000 2018, in_reply_to_user_id_str=null, source=TweetDeck, retweet_count=73, retweeted=false, geo=null, in_reply_to_screen_name=null, is_quote_status=false, id_str=952563458616172544, in_reply_to_user_id=null, favorite_count=186, id=9.5256345861617254E17, text=#UltimOra, “Ai piedi le Clarks”; #LiberieUguali trova il primo punto di convergenza fra i vari leader https://omissis, place=null, lang=it, favorited=false, possibly_sensitive=false, coordinates=null, truncated=false, entities={urls=[Ljava.lang.Object;#7d41de2d, hashtags=[Ljava.lang.Object;#5e05507d, user_mentions=[Ljava.lang.Object;#7b5572f2, symbols=[Ljava.lang.Object;#fb373c9}, contributors=null, user={utc_offset=3600, friends_count=25, profile_image_url_https=https://pbs.twimg.com/profile_images/2932719853/4bcd23b39293870bf1832aec72b6fe3d_normal.png, listed_count=712, profile_background_image_url=http://pbs.twimg.com/profile_background_images/696203701/001bc779080491195bc16054af629016.png, default_profile_image=false, favourites_count=385, description=Dal 13 aprile 2017 in fumetteria e libreria con "Lercio, lo sporco che fa notizia. Il libro" (Shockdom), created_at=Sun Oct 28 18:19:33 +0000 2012, is_translator=false, profile_background_image_url_https=https://pbs.twimg.com/profile_background_images/696203701/001bc779080491195bc16054af629016.png, protected=false, screen_name=lercionotizie, id_str=910827588, profile_link_color=9B1212, is_translation_enabled=false, translator_type=none, id=910827588, geo_enabled=false, profile_background_color=FFFFFF, lang=it, has_extended_profile=false, profile_sidebar_border_color=000000, profile_text_color=333333, verified=false, profile_image_url=http://pbs.twimg.com/profile_images/2932719853/4bcd23b39293870bf1832aec72b6fe3d_normal.png, time_zone=Amsterdam, url=http://omissis, contributors_enabled=false, profile_background_tile=false, profile_banner_url=https://pbs.twimg.com/profile_banners/910827588/1467828171, entities={description={urls=[Ljava.lang.Object;#7c7074a4}, url={urls=[Ljava.lang.Object;#80245ef}}, statuses_count=7335, follow_request_sent=false, followers_count=491614, profile_use_background_image=true, default_profile=false, following=true, name=lercio-notizie, location=, profile_sidebar_fill_color=DDEEF6, notifications=false}}
What's wrong?
You are probably using Logger.log for checking results of code execution.
As noted in documentation:
The method Logger.log expects a string value, but if you pass in an object or array it will do its best to convert that into a string.
So response in this case response=response.getContentText() will give you String with expected behaviour when used in Logger.log, but in case response=JSON.parse(response) will give Object, that will have output in Logger.log in format [key1=val1, key2=val2, ..].
You can get used to it, or you can use convertion of your object to string before logging it:
Logger.log(JSON.stringify(response))