Why am I not being able of detecting a None value from a dictionary - json

I have seen this issue many times happening to many people (here). I am still struggling trying to validate whether what my dictionary captures from a JSON is "None" or not but I still get the following error.
This code is supposed to call a CURL looking for the 'closed' value in the 'status' key until it finds it (or 10 times). When payment is done by means of a QR code, status changes from opened to closed.
status = (my_dict['elements'][0]['status'])
TypeError: 'NoneType' object is not subscriptable
Any clue of what am I doing wrong and how can I fix it?
Also, if I run the part of the script that calls the JSON standalone, it executes smoothly everytime. Is it anything in the code that could be affecting the CURL execution?
By the way, I have started programming 1 week ago so please excuse me if I mix concepts or say something that lacks of common sense.
I have tried to validate the IF with "is not" instead of "!=" and also with "None" instead of "".
def show_qr():
reference_json = reference.replace(' ','%20') #replaces "space" with %20 for CURL assembly
url = "https://api.mercadopago.com/merchant_orders?external_reference=" + reference_json #CURL URL concatenate
headers = CaseInsensitiveDict()
headers["Authorization"] = "Bearer MY_TOKEN"
pygame.init()
ventana = pygame.display.set_mode(window_resolution,pygame.FULLSCREEN) #screen settings
producto = pygame.image.load("qrcode001.png") #Qr image load
producto = pygame.transform.scale(producto, [640,480]) #Qr size
trials = 0 #sets while loop variable start value
status = "undefined" #defines "status" variable
while status != "closed" and trials<10: #to repeat the loop until "status" value = "closed"
ventana.blit(producto, (448,192)) #QR code position setting
pygame.display.update() #
response = requests.request("GET", url, headers=headers) #makes CURL GET
lag = 0.5 #creates an incremental 0.5 seconds everytime return value is None
sleep(lag) #
json_data = (response.text) #Captures JSON response as text
my_dict = json.loads(json_data) #creates a dictionary with JSON data
if json_data != "": #Checks if json_data is None
status = (my_dict['elements'][0]['status']) #If json_data is not none, asigns 'status' key to "status" variable
else:
lag = lag + 0.5 #increments lag
trials = trials + 1 #increments loop variable
sleep (5) #time to avoid being banned from server.
print (trials)

From your original encountered error, it's not clear what the issue is. The problem is that basically any part of that statement can result in a TypeError being raised as the evaluated part is a None. For example, given my_dict['elements'][0]['status'] this can fail if my_dict is None, or also if my_dict['elements'] is None.
I would try inserting breakpoints to better assist with debugging the cause. another solution that might help would be to wrap each part of the statement in a try-catch block as below:
my_dict = None
try:
elements = my_dict['elements']
except TypeError as e:
print('It possible that my_dict maybe None.')
print('Error:', e)
else:
try:
ele = elements[0]
except TypeError as e:
print('It possible that elements maybe None.')
print('Error:', e)
else:
try:
status = ele['status']
except TypeError as e:
print('It possible that first element maybe None.')
print('Error:', e)
else:
print('Got the status successfully:', status)

Related

ReadProcessMemory() dont read pages with specific AllocationProtect values

I'm building a Memory Scanner and with some error handling I noticed that ReadProcessMemory() is reading 90% of process' pages, but the ones that have mbi.Protect value == 1 or 260 it fails and returns ERROR 299 (Partial Copy) and the output of BytesRead is 0.
I run it as admin, set debug privileges and open process with VM_READ, but these exactly pages with mbi.Protect == 260 and 1 are unreadable. So, it's normal that it cant read all pages or am I doing something wrong ? Here is the code: (to be reproducable it also need this part of code that I import to main code and its where I setup all the ctypes background: https://pastebin.com/hMxLej5k, then you open python, import the code below and write "main(pid)" where pid is the pid of the process you want to read).
from ctypes import *
from ctypes import wintypes
import win32security
from setup_apis import *
def setDebugPriv():
token_handle = wintypes.HANDLE()
if not OpenProcessToken(
GetCurrentProcess(),
TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY,
byref(token_handle),
):
print("Error:",kernel32.GetLastError())
return False
luidvalue = win32security.LookupPrivilegeValue ( None, win32security.SE_DEBUG_NAME )
if not win32security.LookupPrivilegeValue(
None,
win32security.SE_DEBUG_NAME ,
):
print("Error",kernel32.GetLastError())
return False
se_debug_name_value = LUID(luidvalue) # Valor local do Privilégio de Debug
LAA = LUID_AND_ATTRIBUTES (
se_debug_name_value,
SE_PRIVILEGE_ENABLED
)
tkp = TOKEN_PRIVILEGES (
1, # DWORD PrivilegeCount
LAA, # LUID_AND_ATTRIBUTES
)
if not AdjustTokenPrivileges(
token_handle,
False,
byref(tkp),
sizeof(tkp),
None,
None,
):
print("Error:",GetLastError)
CloseHandle(token_handle)
return False
return True
#################################
def main(pid=None):
setDebugPriv()
process = OpenProcess (
PROCESS_VM_READ|PROCESS_QUERY_INFORMATION,
False,
pid,
)
system_info = SYSTEM_INFO()
GetSystemInfo ( byref(system_info) )
MaxAppAdress = system_info.lpMaximumApplicationAdress
VirtualQueryEx = VirtualQueryEx64
mbi = MEMORY_BASIC_INFORMATION64()
memset (
byref(mbi),
0,
sizeof(mbi),
)
Adress = 0
BytesRead = c_size_t (0)
while MaxAppAdress > Adress:
VirtualQueryEx(
process,
Adress,
byref(mbi),
sizeof(mbi),
)
if mbi.State == MEM_COMMIT:
try:
ContentsBuffer = create_string_buffer(mbi.RegionSize)
except:
pass
if not ReadProcessMemory (
process,
Adress,
ContentsBuffer,
mbi.RegionSize,
byref(BytesRead),
):
print("Cant Read, Error: %i, Protect State: %i" %(kernel32.GetLastError(), mbi.Protect) )
print("BytesRead:", BytesRead)
Adress += mbi.RegionSize
continue
Adress += mbi.RegionSize
'''
See Memory Protection Constants (260 = 0x104). No access and page guard regions cause exceptions. You can't access a no_access and you don't want to fire page_guard exceptions as they are meant to warn a process that a stack needs to grow and commit more pages. Don't attempt to read them.
Constant
Value
Description
PAGE_NOACCESS
0x01
Disables all access to the committed region of pages. An attempt to read from, write to, or execute the committed region results in an access violation.This flag is not supported by the CreateFileMapping function.
PAGE_READWRITE
0x04
Enables read-only or read/write access to the committed region of pages. If Data Execution Prevention is enabled, attempting to execute code in the committed region results in an access violation.
PAGE_GUARD
0x100
Pages in the region become guard pages. Any attempt to access a guard page causes the system to raise a STATUS_GUARD_PAGE_VIOLATION exception and turn off the guard page status. Guard pages thus act as a one-time access alarm. For more information, see Creating Guard Pages.When an access attempt leads the system to turn off guard page status, the underlying page protection takes over.If a guard page exception occurs during a system service, the service typically returns a failure status indicator.This value cannot be used with PAGE_NOACCESS.This flag is not supported by the CreateFileMapping function.

Cannot index into null array from function returned variable, or issues accessing regex data returned

I'm not sure if I'm returning the value from the function incorrectly, but when I try to access it's info, it has the above error,
Cannot index into a null array
I've tried a couple different ways, and I'm not sure if I'm not returning this correctly from the function, or if I'm just accessing the info returned incorrectly. Looking at Cannot index into null array, it looks like for him, some of his array had null values. But when I print my info to screen before I exit the function, it has info. How do I return the value found in the function such that I can loop through the contents in my main code and use one of the strings in the object? This is a continuation of parsing repeated pattern.
#parse data out of cpp code and loop through to further process
#function
Function Get-CaseContents{
[cmdletbinding()]
Param ( [string]$parsedCaseMethod, [string]$parseLinesGroupIndicator)
Process
{
# construct regex
$fullregex = [regex]"_stprintf[\s\S]*?_T\D*", # Start of error message, capture until digits
"(?<sdkErr>\d+)", # Error number, digits only
"\D[\s\S]*?", # match anything, non-greedy
"(?<sdkDesc>\((.+?)\))", # Error description, anything within parentheses, non-greedy
"([\s\S]*?outError\s*=(?<sdkOutErr>\s[a-zA-Z_]*))", # Capture OutErr string and parse out part after underscore later
"[\s\S]*?", # match anything, non-greedy
"(?<sdkSeverity>outSeverity\s*=\s[a-zA-Z_]*)", # Capture severity string and parse out part after underscore later
'' -join ''
# run the regex
$Values = $parsedCaseMethod | Select-String -Pattern $fullregex -AllMatches
# Convert Name-Value pairs to object properties
$result = foreach ($match in $Values.Matches){
[PSCustomObject][ordered]#{
sdkErr = $match.Groups['sdkErr']
sdkDesc = $match.Groups['sdkDesc']
sdkOutErr = $match.Groups['sdkOutErr']
sdkSeverity = ($match.Groups['sdkSeverity'] -split '_')[-1] #take part after _
}
}
#Write-Host "result:" $result -ForegroundColor Green
$result
return $Values
...
#main code
...
#call method to get case info (sdkErr, sdkDesc, sdkOutErr, sdkSeverity)
$ValuesCase = Get-CaseContents -parsedCaseMethod $matchFound -parseLinesGroupIndicator "_stprintf" #need to get returned info back
$result = foreach ($match in $ValuesCase.Matches){
[PSCustomObject][ordered]#{
sdkErr = $match.Groups['sdkErr']
sdkDesc = $match.Groups['sdkDesc']
sdkOutErr = $match.Groups['sdkOutErr']
sdkSeverity = ($match.Groups['sdkSeverity'] -split '_')[-1] #take part after _
} #result
} #foreach ValuesCase
The example of string sent to the function to parse is:
...
case kRESULT_STATUS_Undefined_Opcode:
_stprintf( outDevStr, _T("8004 - (Comm. Err 04) - %s(Undefined Opcode)"), errorStr);
outError = INVALID_PARAM;
outSeverity = CCA_WARNING;
break;
case kRESULT_STATUS_Comm_Timeout:
_stprintf( outDevStr, _T("8005 - (Comm. Err 05) - %s(Timeout sending command)"), errorStr);
outError = INVALID_PARAM;
outSeverity = CCA_WARNING;
break;
case kRESULT_STATUS_TXD_Failed:
_stprintf( outDevStr, _T("8006 - (Comm. Err 06) - %s(TXD Failed--Send buffer overflow.)"), errorStr);
outError = INVALID_PARAM;
outSeverity = CCA_WARNING;
break;
...
Another thing I tried is (but it also had the index into null array issue):
foreach($matchRegex in $ValuesCase.Matches)
{
$sdkOutErr = $matchRegex.Groups['sdkOutErr']
Write-Host sdkOutErr -ForegroundColor DarkMagenta
}
Ultimately, I need to grab $sdkOutErr to further process. I'll need to use the other variables too in the returned object, but this is the first one I need. I love the way the output is formatted in the function, but probably don't know how to return the info and use what is returned. I'm not sure what to search for to figure out the issue other than the error message, which leads me to believe I'm returning the info wrong. I don't think I need to return $result, because I think that's just a string with the values in the $values.Matches in the function. I need to access the values returned as I mentioned.
I checked, and the contents sent to the function is not blank.
I tried returning $results, and it looks like this when I write-Host, which would be difficult to access each sdkOutErr:
#{sdkErr=1000; sdkDesc=(Out of Memory); sdkOutErr= NO_MEMORY; sdkSeverity=FATAL} #{sdkErr=1002; sdkDesc=(Failed to load DLL); sdkOutErr= OTHER_ERROR; sdkSeverity=FATAL} #{sdkErr=1003; sdkDesc=(Failed to load DLL); sdk
OutErr= OTHER_ERROR; sdkSeverity=FATAL} #{sdkErr=1004; sdkDesc=(Failed to open); sdkOutErr= OTHER_ERROR; sdkSeverity=FATAL} #{sdkErr=1005; sdkDesc=(Unable to access the specified profile); sdkOutErr= OTHER_ERROR; sdkSeverity=
FATAL} #{sdkErr=100 ...
How can I return this from the function so that it's not a null array/index, and the data is accessible if I use a foreach loop (or two) in the main code to get the sdkOutErr (to start).
I'm fairly new to (complicated)powershell and I have a feeling I need a map inside the array in my function, but I'm not sure.
Before I returned the function Values or results, it was printing something like this out. Once I added in main $ValuesCase=Get-CaseContents... (returning $values from function), or $parsedCase = Get-CaseContents... (returning $results from function), it stopped showing this on the screen:
sdkErr sdkDesc sdkOutErr sdkSeverity
------ ------- --------- -----------
1000 (Out of Memory) NO_MEMORY FATAL
1002 (Failed to load DLL) OTHER_ERROR FATAL
1003 (Failed to load DLL) OTHER_ERROR FATAL
1004 (Failed to open) OTHER_ERROR FATAL
I tried returning $results, and it looks like this when I write-Host, which would be difficult to access each sdkOutErr:
Getting all the sdkOutErr values is not as difficult as you might imagine:
$results.sdkOutErr # this will output the `sdkOutErr` value from each object in the array
Or, outside the function:
(Get-CaseContents -parsedCaseMethod $matchFound -parseLinesGroupIndicator "_stprintf").sdkOutErr
Another option, which might perform better if the result set is large, is to use ForEach-Object to grab just the sdkOutErr values:
$fullResults = Get-CaseContents -parsedCaseMethod $matchFound -parseLinesGroupIndicator "_stprintf"
$sdkOutErrValuesOnly = $fullResults |ForEach-Object -MemberName sdkOutErr

Ruby: Handling different JSON response that is not what is expected

Searched online and read through the documents, but have not been able to find an answer. I am fairly new and part of learning Ruby I wanted to make the script below.
The Script essentially does a Carrier Lookup on a list of numbers that are provided through a CSV file. The CSV file has just one row with the column header "number".
Everything runs fine UNTIL the API gives me an output that is different from the others. In this example, it tells me that one of the numbers in my file is not a valid US number. This then causes my script to stop running.
I am looking to see if there is a way to either ignore it (I read about Begin and End, but was not able to get it to work) or ideally either create a separate file with those errors or just put the data into the main file.
Any help would be much appreciated. Thank you.
Ruby Code:
require 'csv'
require 'uri'
require 'net/http'
require 'json'
number = 0
CSV.foreach('data1.csv',headers: true) do |row|
number = row['number'].to_i
uri = URI("https://api.message360.com/api/v3/carrier/lookup.json?PhoneNumber=#{number}")
req = Net::HTTP::Post.new(uri)
req.basic_auth 'XXX' , 'XXX'
res = Net::HTTP.start(uri.hostname, uri.port, :use_ssl => true) {|http|
http.request(req)
}
json = JSON.parse(res.body)
new = json["Message360"]["Carrier"].values
CSV.open("new.csv", "ab") do |csv|
csv << new
end
end
File Data:
number
5556667777
9998887777
Good Response example in JSON:
{"Message360"=>{"ResponseStatus"=>1, "Carrier"=>{"ApiVersion"=>"3", "CarrierSid"=>"XXX", "AccountSid"=>"XXX", "PhoneNumber"=>"+19495554444", "Network"=>"Cellco Partnership dba Verizon Wireless - CA", "Wireless"=>"true", "ZipCode"=>"92604", "City"=>"Irvine", "Price"=>0.0003, "Status"=>"success", "DateCreated"=>"2018-05-15 23:05:15"}}}
The response that causes Script to stop:
{
"Message360": {
"ResponseStatus": 0,
"Errors": {
"Error": [
{
"Code": "ER-M360-CAR-111",
"Message": "Allowed Only Valid E164 North American Numbers.",
"MoreInfo": []
}
]
}
}
}
It would appear you can just check json["Message360"]["ResponseStatus"] first for a 0 or 1 to indicate failure or success.
I'd probably add a rescue to help catch any other errors (malformed JSON, network issue, etc.)
CSV.foreach('data1.csv',headers: true) do |row|
number = row['number'].to_i
...
json = JSON.parse(res.body)
if json["Message360"]["ResponseStatus"] == 1
new = json["Message360"]["Carrier"].values
CSV.open("new.csv", "ab") do |csv|
csv << new
end
else
# handle bad response
end
rescue StandardError => e
# request failed for some reason, log e and the number?
end

World of tanks Python list comparison from json

ok I am trying to create a definition which will read a list of IDS from an external Json file, Which it is doing. Its even putting the data into the database on load of the program, my issue is this. I cant seem to match the list IDs to a comparison. Here is my current code:
def check(account):
global ID_account
import json, httplib
if not hasattr(BigWorld, 'iddata'):
UID_DB = account['databaseID']
UID = ID_account
try:
conn = httplib.HTTPConnection('URL')
conn.request('GET', '/ids.json')
conn.sock.settimeout(2)
resp = conn.getresponse()
qresp = resp.read()
BigWorld.iddata = json.loads(qresp)
LOG_NOTE('[ABRO] Request of URL data successful.')
conn.close()
except:
LOG_NOTE('[ABRO] Http request to URL problem. Loading local data.')
if UID_DB is not None:
list = BigWorld.iddata["ids"]
#print (len(list) - 1)
for n in range(0, (len(list) - 1)):
#print UID_DB
#print list[n]
if UID_DB == list[n]:
#print '[ABRO] userid located:'
#print UID_DB
UID = UID_DB
else:
LOG_NOTE('[ABRO] userid not set.')
if 'databaseID' in account and account['databaseID'] != UID:
print '[ABRO] Account not active in database, game closing...... '
BigWorld.quit()
now my json file looks like this:
{
"ids":[
"1001583757",
"500687699",
"000000000"
]
}
now when I run this with all the commented out prints it seems to execute perfectly fine up till it tries to do the match inside the for loop. Even when the print shows UID_DB and list[n] being the same values, it does not set my variable, it doesn't post any errors, its just simply acting as if there was no match. am I possibly missing a loop break? here is the python log starting with the print of the length of the table print:
INFO: 2
INFO: 1001583757
INFO: 1001583757
INFO: 1001583757
INFO: 500687699
INFO: [ABRO] Account not active, game closing......
as you can see from the log, its never printing the User located print, so it is not matching them. its just continuing with the loop and using the default ID I defined above the definition. Anyone with an idea would definitely help me out as ive been poking and prodding this thing for 3 days now.
the answer to this was found by #VikasNehaOjha it was missing simply a conversion to match types before the match comparison I did this by adding in
list[n] = int(list[n])
that resolved my issue and it finally matched comparisons.

How do I get all tweets of a user?

I'm trying to get all of a specific user's tweets.
I know there is a limit of retreiving 3600 tweets, so I'm wondering why I can't get more tweets from this line:
https://api.twitter.com/1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=mybringback&count=3600
Does anyone know how to fix this?
The API documentation specifies that the maximum number of statuses that this call will return is 200.
https://dev.twitter.com/docs/api/1/get/statuses/user_timeline
Specifies the number of tweets to try and retrieve, up to a maximum of 200. The value of count is best thought of as a limit to the number of tweets to return because suspended or deleted content is removed after the count has been applied. We include retweets in the count, even if include_rts is not supplied. It is recommended you always send include_rts=1 when using this API method.
Here's something I've used for a project that had to do just that:
import json
import commands
import time
def get_followers(screen_name):
followers_list = []
# start cursor at -1
next_cursor = -1
print("Getting list of followers for user '%s' from Twitter API..." % screen_name)
while next_cursor:
cmd = 'twurl "/1.1/followers/ids.json?cursor=' + str(next_cursor) + \
'&screen_name=' + screen_name + '"'
(status, output) = commands.getstatusoutput(cmd)
# convert json object to dictionary and ensure there are no errors
try:
data = json.loads(output)
if data.get("errors"):
# if we get an inactive account, write error message
if data.get('errors')[0]['message'] in ("Sorry, that page does not exist",
"User has been suspended"):
print("Skipping account %s. It doesn't seem to exist" % screen_name)
break
elif data.get('errors')[0]['message'] == "Rate limit exceeded":
print("\t*** Rate limit exceeded ... waiting 2 minutes ***")
time.sleep(120)
continue
# otherwise, raise an exception with the error
else:
raise Exception("The Twitter call returned errors: %s"
% data.get('errors')[0]['message'])
if data.get('ids'):
print("\t\tFound %s followers for user '%s'" % (len(data['ids']), screen_name))
followers_list += data['ids']
if data.get('next_cursor'):
next_cursor = data['next_cursor']
else:
break
except ValueError:
print("\t****No output - Retrying \t\t%s ****" % output)
return followers_list
screen_name = 'AshwinBalamohan'
followers = get_followers(screen_name)
print("\n\nThe followers for user '%s' are:\n%s" % followers)
In order to get this to work, you'll need to install the Ruby gem 'Twurl', which is available here: https://github.com/marcel/twurl
I found Twurl easier to work with than the other Python Twitter wrappers, so opted to call it from Python. Let me know if you'd like me to walk you through how to install Twurl and the Twitter API keys.