I'm currently developing a GUI using Python and Tkinter.
On of the task is to open and read some *.csv files.
I order to perform this task I have written the following code:
ReadData=pd.read_csv(ResultFile,skipinitialspace=True).values
While I'm running the code within the IDE Eclipse+Pydev everything work fine. But as soon as I run my code form a Dos window, i.e. python MainGrap.py, the code bugs stating that the file doesn't exists???????
I first load the path to a file via self.Inp_Filename=askopenfilename() then I create a list of the folders by means of the following function:
def PathDisintegrator(Inp_File):
Folders = os.path.split(Inp_File)
LastFolder = Folders[1]
RootPath = Folders[0]
Dirs=[]
while not(LastFolder==''):
Dirs.insert(0,LastFolder)
Folders = os.path.split(RootPath)
LastFolder = Folders[1]
RootPath = Folders[0]
Dirs.insert(0,RootPath[:-1])
Dirs=Dirs[:-1]
return(Dirs)
Then I can recreate the full path to file via the following function:
def PathAndFile(Folders,File):
FileOut=''
for item in Folders:
FileOut=FileOut+os.sep+item
#FileOut=FileOut+r"\\"+item
FileOut=FileOut[1:]+os.sep+File
return(FileOut)
I have printed out the file path even within the parser of Pandas and it looks fine to me: D:\Abaqus_Runs\DOWLEX_PET_LAMINATE_PROTO_REFERENCE_SI_Version_2_Revision_2_MDangle0_Rate0_01_MOVING_NODE_out.csv
The problem here is that your python environment in eclipse can see the folder where your csv resides but the terminal one does not.
You can observe what the system paths are by doing:
In [331]:
import sys
sys.path
Out[331]:
['',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\python34.zip',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\DLLs',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib\\site-packages',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib\\site-packages\\win32',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib\\site-packages\\win32\\lib',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib\\site-packages\\Pythonwin',
'C:\\WinPython-64bit-3.4.2.4\\python-3.4.2.amd64\\lib\\site-packages\\IPython\\extensions']
So you need to provide a complete path or append the path to where the csv resides to your sys path. Note that backslashes must be escaped e.g. 'c:\\data\\my.csv' but if you use forward slashes then it works fine: e.g. 'c:/data/my.csv'
Related
I was trying to use couchbase in one of my android applications. This app will have static data that are scraped from the web. So I want to generate a cblite2 file with my python script and insert those data and then use this cblite2 file in android. I can load data from an existing file according to this. But how can I generate my initial cblite2 file?
You could use the cblite command line tool to create the database. There are a couple of ways to do the import. I'll describe what seems to me like the simplest way.
Have your script save the JSON documents to a directory. For this example, let's call the diretory json-documents. Use the desired document ID as the base name, and .json as the extension. For example, if you want a document's ID to be "foo", the filename would be foo.json.
Then use the cblite cp command to import the documents into a new database:
cblite cp json-documents/ myNewDatabase.cblite2
You can use the other cblite subcommands to verify the import was successful.
List the document IDs:
cblite ls myNewDatabase.cblite2
Display the contents of a document:
cblite cat myNewDatabase.cblite2 someDocumentId
I have a Python script that does some web-scraping, then opens and dumps the parsed data into a JSON file in the same directory. Everything works, when the script is run manually through the CLI, but the data does not get written to the JSON file, when run from the batch file run by a task scheduler.
I have managed to show that all the data exists within the Python script, when run through the batch file. Somehow only part of the function that deals with the JSON file is not run.
Python script:
# Packages used:
import requests
from bs4 import BeautifulSoup
import smtplib
import time
from win10toast import ToastNotifier
import json
# Web Scraping...
my_json = {}
def function1():
# Web scraping for data...
json_function(data)
# Below is the function that is not functioning
def json_function(data):
my_json[time.strftime("%Y-%m-%d %H:%M")] = f"{data}"
with open ('json_file.json') as my_dict:
info = json.load(my_dict)
info.update(my_json)
with open('json_file.json','w') as my_dict:
json.dump(info,my_dict)
# A few other functions that work regardless...
# Call function
function1()
Batch file:
"C:\Users\...pythonw.exe" "C:\Users...script.pyw"
JSON file:
{"Key":"Value"}
Every file is in the same directory.
When run from the CLI, expected result occurs - key-value are appended to the JSON file. When run automatically (through batch and task scheduler), no visible errors, and all of the script, save for the json_function, run as expected.
Thank you to #PRMoureu for the answer, and #Mofi for a detailed explanation.
Answer is to ensure all files referenced have their full path referenced:
def json_function(data):
my_json[time.strftime("%Y-%m-%d %H:%M")] = f"{data}"
with open ('C:/.../json_file.json') as my_dict:
info = json.load(my_dict)
info.update(my_json)
with open('C:/.../json_file.json','w') as my_dict:
json.dump(info,my_dict)
Or, direct the Task Scheduler to the working directory to avoid the batch being run in the default, root directory.
i am building an extension to parse json using vs code extension.
so my need is ,it should be able able to load .json file from a particular folder and iterate through content of the file.
Then it should allow user to select few keys from it make a new json file out of this and save it in any folder.
But i am not able to find any way to read and write files in "vs code extension".Could someone please help me.
If you want to read the current edit state of a file you can use the following API workspace function:
vscode.workspace.openTextDocument(uri).then((document) => {
let text = document.getText();
});
This will show you the current state of the file including unpersisted changes. document is of type TextDocument and has isDirty set to true if it has pending changes.
Since the extension runs in nodejs, you should be able to use any nodejs module built-in or installed by npm in the usual way.
For your purpose you will be OK with the built-in fs module: https://nodejs.org/dist/latest-v6.x/docs/api/fs.html
In your extension you will need to import the required module, so your code file should contain this:
let fs = require("fs");
and then use the methods in the usual way, eg. fs.fileReadSync( filename, encoding ) ...
Please not that there is one exception. If you install a nodejs module containing compiled, binary code, it will not run in the extension and instead you will see an error message saying something like %1 is not a valid Win32 application. Pure javascript modules are OK, though.
VSCode extensions are running in node.js. Therefore you can use any available node.js package/module within your extension. For instance, check out this question for reading JSON.
For JSON, you just need to require or import the JSON file, such as:
const jsonObject = require('./myJSONfile.json');
// do something
For JSON with comments, you can use node-jsonc-parser.
After the manipulation, you could use the fs module of nodej.js to write to the disk.
For a course on Excel I was trying to load a CSV in Neo4j (first time using this application) when I was blocked at the first step of replicating an example shown in said course: loading.
The command which was used in the example was this;
LOAD CSV WITH HEADERS FROM "file:/path/to/file/file.csv"
as row
CREATE (m:movie {name:row.movie})
But it gave syntax errors. I found out I could correct it by using double \ and add "file:";
LOAD CSV WITH HEADERS FROM "file://C:\\path\\to\\file\\file.csv"
as row
CREATE (m:movie {name:row.movie})
Neo4j accepts this syntax, processes for a few moments, and returns YET ANOTHER error;
Neo.TransientError.Statement.ExternalResourceFailure
I tried the same commands (original and my own) in the online Neo4j console but no luck. I can reach the file using that path without problem; it really is there. The CSV file consist out of just 5 strings of regular letters, that's all. No fancy formatting or characters.
What's going on?
Not that mysterious, Neo4j's IMPORT CSV function looks for the specified CSV file in the import directory within your server configuration for that database, as specified at the top of its server configuration file. (IE: dbms.directories.import=import in your neo4j.conf file.)
You should create the import directory in...
"C:\Users\[User Name]\Documents\Neo4j\default.graphdb\"
If you place your CSV file in there, you can specify any sub-directory or just the "file.csv" you want to import with the IMPORT CSV function as below.
LOAD CSV WITH HEADERS FROM "file:///file.csv"
AS row
RETURN row
LIMIT 5
Try using:
"file:///C:/path/to/file/file.csv"
Since your file is on your local computer, the third / following the file scheme is not preceded by a host name or address -- but it still needs to be there. Also, file URI path separators should be forward slashes (even on Windows machines).
See the File URI scheme Wikipedia page if you need more information.
I am trying to complete an assignment in Python3. It is very similar to the pdf found here
I have a few questions on both the execution of how to get the information I need, and if possible, some code that could move me along. I am new to python. As right now from the code I have, I keep getting the error "directory not found" after running a function to try and read the data. I know the .csv file should be in the directory where I save it to in WingIDE, but I can't get it to work correctly.
My first question is after getting each line of the .csv file to read from my get_file_list, what is the best way to take each category and throw it into an efficiency equation?
Here is my get_data_list function:
def get_data_list(filename):
data_file = open(filename, "r")
data_list = [ ]
for line_str in data_file:
data_list.append(line_str.strip().split(','))
return data_list
when I run get_data_list("player_regular_season.csv") I get the following error:
builtins.IOError: [Errno 2] No such file or directory:'player_regular_season.csv'
For the first try, you can put the data file to the same directory with the Python program and launch it from the directory.
Try also a single purpose script to learn how to work with directories. Learn the functions from the standard doc 15.1.5. Files and Directories, namely os.getcwd(), os.chdir(path), and then 10.1. os.path — Common pathname manipulations, namely os.path.isfile(path).
But read also the doc of other functions in the documents to learn what is available.
When knowing how to work with filenames and paths, have a look at the 13.1. csv — CSV File Reading and Writing. Not to be scared of all the stuff, start from the end -- 13.1.5. Examples of using the csv module.