Parameters: I'm in directory named algorithm and this algorithm is in a parent directory called src. There is another directory called data that is also in the src parent directory. E.g. the relative path of data is src/data and the relative path of algorithm is src/algorithm. There is also an info.csv file in the data directory.
If I wanted to read the contents of info.csv in a program currently located in the data subdirectory, how would I do that? Doing something like CSV.File("..//data//info.csv"; delim = ";") does not appear to be working.
If you are using a REPL-notebook style workflow why not utilise the functionality of the REPL:
; # To enter shell mode
mv /your/path/file /destination/path/file #or cp if you want a copy
# backspace to return to Julia REPL
The relative path in CSV.File("..//data//info.csv"; delim = ";") is interpreted relative to the current directory path, set by the shell/OS. If you want it to be interpreted relative to the path where the script happens to be located (i.e. the algorithm directory), you can explicitly do that using #__DIR__:
#__DIR__ -> AbstractString
Expand to a string with the absolute path to the directory of the file containing the macrocall.
csvfilepath = joinpath(#__DIR__, "..", "data", "info.csv")
CSV.File(csvfilepath; delim = ";")
Related
I am creating a function that copies a folder and pastes it with another name.
Therefore, I have my "Main" folder and according to the query I make copies of the folder called like this: "Main 1", "Main 2", "Main 3"
The way I managed to solve this was through this function:
function TDMMonitor.CopyFolder(Origin, Destination : String) : Boolean;
var
aFiles : TStringDynArray;
InFile, OutFile: string;
Begin
aFiles := TDirectory.GetFiles(Origin, '*.*', TSearchOption.soAllDirectories);
for InFile in aFiles do
Begin
OutFile := TPath.Combine(Destination , TPath.GetFileName(InFile));
TFile.Copy(InFile, OutFile, True);
End;
Result := True;
End;
This works! But my problem right now is that the parent folder has subfolders that are not being copied correctly.
I leave a more visual example of the results of my function below:
"Main" folder:
File.txt
File1.txt
Sub-Folder -> File3.txt
"Main 1" folder:
File.txt
File1.txt
File3.txt
How can I maintain the folder structure that the Main folder follows?
TDirectory.Copy works:
TDirectory.Copy('D:\Path\Main', 'D:\Path\Main 1');
TDirectory.GetFiles() returns an array of absolute paths to each file found. The soAllDirectories flag tells it to search through subfolders recursively. So, you will end up with an array of paths at potentially different levels.
TPath.GetFileName() strips off all folder path info, leaving just the file name.
Based on your example, you are searching recursively through C:\Main\ and you want to copy everything to C:\Main 1\. When the search gives you the file C:\Main\Sub-Folder\File3.txt, your use of TPath.GetFileName() discards C:\Main\Sub-Folder\, and so you concatenate C:\Main 1\ with just File3.txt, thus you copy C:\Main\Sub-Folder\File3.txt to C:\Main 1\File3.txt instead of to C:\Main 1\Sub-Folder\File3.txt.
That is why your subfolders are not copying correctly.
Rather than using TPath.GetFileName(), you would need to replace only the Origin portion of each returned absolute path (C:\Main\) with the Destination path (C:\Main 1\), leaving everything else in the path intact. Thus, C:\Main\Sub-Folder\File3.txt would become C:\Main 1\Sub-Folder\File3.txt.
Otherwise, don't use soAllDirectories at all. Recurse through the subfolders manually using TDirectory.GetDirectories() instead, so that you are handling only 1 level of folders at a time.
How can I get the contents of a directory containing XML files in Octave and store the file names in an Array?
There is dir, ls, readdir and glob. If you want to search a pattern I would suggest glob:
fns = glob ("*.xlm");
which will return a cell array with relative (to pwd) filenames.
I am reading CSV files from datalake store, for that I am having multiple paths but if any one path does not exist it gives exception. I want to avoid this expection.
I think if you want to check for multiple pathes, the check will fail if one path does not exist. Perhaps you could try a different approach.
For the given example if you want to subselect subfolders you could try the following instead.
Read sub-directories of a given directory:
# list all subfolders and files in directory demo
dir = dbutils.fs.ls ("/mnt/adls2/demo")
Filter out the relevant sub-directories:
pathes = ''
for i in range (0, len(dir)):
subpath = dir[i].path
if '/corr' in subpath or '/deci' in subpath and subpath.startswith ('dbfs:/'): # select dirs to read
pathes = pathes + (dir[i].path) + ' '
# convert the string to a list
pathes = list(pathes.split())
Use the result-list to read the dataframe:
df = (spark.read
.json(pathes))
where path is a specific variable, you can use to check if it exists (scala):
dbutils.fs.ls("/mnt").map(_.name).contains(s"$path/")
you must Use block blobs for a Mount point to work from databricks and be able to see the blobs in it.
I have a variable with file path like:
$file = '/some/file.txt';
or
var file = '/some/file.txt'
To edit file.txt I:
Left click on string.
Do Extend Selection shortcut.
Do Navigate File... shortcut.
Can I do it faster? For example, by clicking on '/some/file.txt' with some modifier key.
Install and use Navigate From Literal plugin -- it works with any strings as it matches files by names.
It's not 100% perfect though .. as it seems to work rather with file names and even though the path in string is pretty unique .. it most likely will show a choice for all files named the same (e.g. file.txt in your case) -- at least this is what I remember when using it.
Assume we have a directory with structure like this, I marked directories as (+) and files as (-)
rootdir
+a
+a1
-f1
-f2
+a2
-f3
+b
+b1
+b2
-f4
-f5
-f6
+b3
-f7
-f8
and a given list of files like
/a/a1/f1
/b/b1/b2/f5
/b/b3/f7
I am struggling to find the way to remove every files inside root, except the one in the given list. So after the program executed, the root directory should look like this:
rootdir
+a
+a1
-f1
+b
+b1
+b2
-f5
+b3
-f7
This example just for easier to understand the problem. In reality, the given list include around 4 thousands of files. And the root directory has the size of ~15GB with a hundreds of thousands files inside.
That would be easy to search inside a folder, and to remove files that matched in a given list. Let just say we solve the revert issue, to keep files that matched in a given list.
Programs written in Perl/Python are prefer.
First, store your list of files you want to keep inside an associative container like a Python dict or a map of some kind.
Second, simply iterate (in Python, os.walk) over the entire directory structure, and every time you see a file, check if it is in the associative container of paths to keep. If not, delete it (in Python, os.unlink).
Alternatively:
First, create a temporary directory on the same filesystem.
Second, move (os.renames, which generates new subdirectories as needed) all the "keep" files to the temporary directory, with the same structure.
Third, overwrite (os.removedirs followed by os.rename, or just shutil.move) the original directory with the temporary one.
The os.walk path:
import os
keep = set(['/a/a1/f1', '/b/b1/b2/f5', '/b/b3/f7'])
for dirpath, dirnames, filenames in os.walk('./'):
for name in filenames:
path = os.path.join(dirpath, name).lstrip('.')
print('check ' + path)
if path not in keep:
print('delete ' + path)
else:
print('keep ' + path)
It doesn't do anything except inform you.
It don't think os.walk is too slow, and it gives you the option of keeping by regex patterns or any other criteria.
This is a working code for your problem.
import os
def list_files(directory):
for root, dirs, files in os.walk(directory):
for name in files:
yield os.path.join(root, name)
files_to_delete = {'/home/vedang/Desktop/a.out', '/home/vedang/Desktop/ABC/temp.txt'} #Keep a set instead of list for faster lookups
for f in list_files('/home/vedang/Desktop'):
if f in files_to_delete:
os.unlink(f)
Here is a function which accepts a set of files you wish to keep and the root directory from which you wish to begin deleting files.
It's a classic recursive Depth-First-Search that will remove empty directories after deleting all the unwanted files
import os
def delete_files(keep_list:set, curr_dir):
files = os.listdir(curr_dir)
for f in files:
path = f"{curr_dir}/{f}"
if os.path.isfile(path):
if path not in keep_list:
os.remove(path)
elif os.path.islink(path):
os.unlink(path)
elif os.path.isdir(path):
delete_files(keep_list, path)
files = os.listdir(curr_dir)
if not files:
os.rmdir(curr_dir)
here i got a solution in a different aspect,
suppose we are at linux environment,
first,
find .
to get a long list with all file path/folder explained
second, suppose we got the exclude path list, in order to exclude at your volume ( say thousands ) , we could just append these to the previous list, and
| sort | uniq - c |grep -v "^2"
to get the to delete list,
and third
| xargs rm
to actually do the deletion