octave mkdir fails after recursive rmdir - octave

I have a code that creates a subfolder but first removes the subfolder if it already existed. I am using Octave3.6.4_gcc4.6.2 for MinGW on a Win7 pro machine. I noticed that mkdir fails if the subfolder existed and contained several files. It seems like rmdir has not completed in the background before the next lines of code are executed. Below is a sample of the test code.
parentDir = 'C:\Temp\rmDir';
childDir = fullfile(parentDir, 'output');
if (exist(childDir, 'dir') ~= 0)
[status] = rmdir(childDir, 's');
disp(status);
end;
[status] = mkdir(parentDir, 'output');
disp(status);
disp(exist(childDir, 'dir'));
Below is the Octave result for when the subfolder does not exist. This works as expected.
octave:5> testrmdir
1
7
Below is the Octave result for when a subfolder exists and is empty. This works as expected.
octave:6> testrmdir
1
1
7
Below is the Octave result for when a subfolder exists and contains 3 PNG files with a total size of 349 KB. Status is 1 for both mkdir and rmdir. However, the exist function reports that the folder does not exist. I confirm from windows explorer that the subfolder is deleted. My guess is that when mkdir executes, the files are still being deleted by the prior rmdir function. So mkdir reports success because the subfolder has not been deleted by rmdir yet. However, by the time exist is executed rmdir has completed and so the subfolder no longer exists.
octave:7> testrmdir
1
1
0
I tried different file types with the following results:
2 PNG files, 232 KB total - pass
4 PNG files, 465 KB total - fail
3 PNG files, 349 KB total - fail
3 csv files, 518 KB total - pass
5 csv files, 777 KB total - fail
The behavior is the same when I run Octave from the command line. I have used the same code on MATLAB in the past without any noticeable issues. For now, I had to switch to Octave for test automation on a different machine.
Does this make sense? Any suggestions on how to make this code work regardless of the subfolder contents or size?
Not sure if this is important, but I have the following setting in the resource file: confirm_recursive_rmdir(false).

I changed the if statement to a while loop and this fixed the problem (i.e. all I did was replace "if" with "while"). Then I added a counter in the while loop and saw that rmdir was successful on the first iteration. Therefore, I cannot explain why the code does not work with an if statement. See expanded code with new counter below. But like I said, the code also works if I simply replace "if" in the original code with "while".
parentDir = 'C:\Temp\rmDir';
childDir = fullfile(parentDir, 'output');
count = 0;
while (exist(childDir, 'dir') ~= 0)
%if (exist(childDir, 'dir') ~= 0)
count++
[status] = rmdir(childDir, 's');
disp(status);
disp(count);
end;
[status] = mkdir(parentDir, 'output');
disp(status);
disp(exist(childDir, 'dir'));

Related

When using "file mkdir" in TCL i get the error that the folder does not exist, but thats why i want to create it..

I am having a little trouble when creating a directory in TCL8.5.9 on a Windows 7 Computer.
set CurrentDir [ file dirname $GUI_DB_path]
set ImageFolderPath [ file join $CurrentDir "DeflectionPlots" ]
# Always try to delete the Folder no matter if it exists or not
file delete -force $ImageFolderPath
# sometimes the following throws an error. Do not understand why.
# Create a clean and empty ImageFolder
file mkdir $ImageFolderPath
Sometimes, but not always i get the error:
cant't create directory.....$ImageFolderPath..... No such file or Directory
Well, that is why I want to create it. Running the code a second time without any changes results in the creation of the Directory as desired. What causes this and how can i resolve the issue? I could catch the error, but then I still would not have my Folder created.
Windows file operations (or their internal locking) is often slow.
I run into problems like yours where deletions/new files/renames take
a while and then I get errors because the file(s) are in some sort of
operating system limbo.
You can add a short sleep between the delete and the create and that
should resolve the issue on Windows.
set ::img_create_sleep 0
after 200 [list set ::img_create_sleep 1]
vwait ::img_create_sleep

Creating Function in Octave

I'm beginning with octave. I've created a file called squareThisNumber.m in My Documents with the following code:
function y = squareThisNumber(x)
y = x^2;
I set the directory to look at My Documents with
cd 'C:\Users\XXXX\My Documents'
I type "squareThisNumber(3)" into octave, and all I'm getting is "Error: 'squareThisNumber' undefined near line 3 column 1." What am I doing wrong?
EDIT:
When I type ls into octave, I get "error: ls: command exited abnormally with status 127". Did I not install Octave correctly?
This behavior sure does seem like there's a problem with octave's current working directory. Does the command dir or pwd also have the same problem?
But you might be able to ignore all of that by
addpath("C:\Users\XXXX\My Documents");
Did you place the end keyword at the end? Code example below works perfectly for me
https://saturnapi.com/fullstack/function-example
% Welcome to Saturn's MATLAB-Octave API.
% Delete the sample code below these comments and write your own!
function y = squareThisNumber(x)
y = x^2;
end
squareThisNumber(9)

SciTe and Python 3 - problems with configuration [UBUNTU]

I have big problem(s) with configuration of SciTE in context of Python 3. I do not know if details have any meaning, so:
[DETAILS]
I downloaded and executed gen_python_3_api.py.
I created folder "api" in usr/share/scite and copy-pasted there python3.api
I edited SciTEUser.properties as written in documentation of gen_python_3_api.py. It did not help a bit, so:
I used more general way found on website of SciTE. I edited python.properties and added a line:
api.$(file.patterns.py)=$(SciteDefaultHome)\api\python.api.
Still no effect.
I just edited another line of python.properties:
if PLAT_GTK
command.go.*.py=python3 -u "$(FileNameExt)"
It finally worked (or I though so).
[/DETAILS]
Now I want to run simple Fibbonaci program that worked well with IDLE.
def Fib(n):
a = 0
b = 1
FibL = []
for i in range (n):
FibL.append(a)
z = a
a = b
b = b+z
return FibL
n = int(input("Number? "))
print(Fib(n))
And I get:
>python3 -u "test.py"
Number? Traceback (most recent call last):
File "test.py", line 38, in <module>
n = int(input("Number? "))
EOFError: EOF when reading a line
>Exit code: 1
I am completely confused. Do somebody know why this things happen and how to fix it?
First of all, the generation of api is for editing only, not running your code.
Resolve the ambiguity with versions by adding full path to command lines (Hope, you do so in the 5th point of the question details)
The problem is in the line:
n = int(input("Number? "))
Here you want input from user, i.e. interactive running, but editor runs commands inside its process and can output simply.
Change your code by adding variable instead of the input command
n=5
or use parameters http://www.scintilla.org/SciTEDoc.html#property-if
Good luck!

How to get SSIS to wait for a file to exist and/or become available

Scenario:
Package#1 creates a flat file that contains multiple messages (one per line) that is dropped in an external system's "INPUT" folder. The file is picked up by the external system and processed and the responses are written in the same format to a file in the "OUTPUT" folder. The file starts to be written while the external system is still processing messages, so it is written as foo.rsppro. When processing is complete and all response messages are written it is renamed foo.rsp.
I need to pick up that file once it is complete (i.e. after the rename) and process using Package#2, which will start immediatly following Package#1. When Package#2 starts, the external system could be in three states:
Processing the first message not yet complete and no response file written yet, in which case I need to wait for foo.rsppro to be written, then renamed to foo.rsp
Processing in progress and foo.rsppro written, in which case I need
to wait for foo.rsppro to be renamed to foo.rsp
Processing completed, foo.rsppro has been written and been renamed to foo.rsp, in which case I just need to process foo.rsp.
I have tried:
using a file in use task but that errors if the expected file isn't present when the task begins (i.e. errors for scenario 1 and 2)
using a file watcher task but that appears to ignore file renames by design, so will never handle scenario 1 or 2
Aside from building a script task, is there a custom task that will handle all three scenarios?
Edit: SSIS 2008 R2
only a script task can help in your case.
consider using FileSystemWatcher within the script if possible or have an application/windows service which can monitor file system using FileSystemWatcher and invoke your packages when the event is triggered.
humm, it seems that you can solve it by using a for each loop container on the output folder and set it to read only .rsp files. That would deal with your .rsp files.
how can scenario 1 and 2 happens if package 2 will only run after package1 is finish? As I understand, package1 renames the file so it will only end when all the files are processes and renamed
EDIT:
ok, no worry, there is a solution for everything.
How about, you create a variable on package1 called #TotalNumberOfFiles with the total number of files to be processed, then you use package one to call pacakge2 (not sure if you are doing this already, but if not is very simple, just use a execute pacakge task) and on package2 you create a "parent package variable" (this is very simple too in case you have never done it) and package 2 just start processing when there are #TotalNumberOfFiles files on the output folder with the .rsp extension?
EDIT2:
I dont know jf there is a command to get that, maybe google it, but if you dont find out you can add a foreachloop container pointing to the output directory and do something like this on a script task:
Public Sub Main()
Dts.Variables("User::filesCount").Value = Dts.Variables("User::FilesCount").Value + 1
Dts.TaskResult = ScriptResults.Success
End Sub
after it finishes counting, just compare with TotalNumberOfFiles. If equal, move to the next task, else sleep for a while and count again
Final code used as follows. Basically loops through until either the file is found or the max specified number of attempts is hit.
Imports System.Threading is required for Thread.sleep. It may not be the most processor efficient method but this is 100% dedicated hardware and the packages is are running in serial.
'loop until number of required attempts is hit or file is found
Do Until iCounter = iAttempts Or bFileFound = True
'Check if the file exists
If File.Exists(sFilename) Then
'Switch bFileFound to true
bFileFound = True
'Report that file has been found to VERIFY_Input_File_Exists_INT variable
Dts.Variables("VERIFY_Input_File_Exists_INT").Value = True
Dts.Events.FireInformation(1, "DEBUG:", sFilename & " found successfully.", "", 0, False)
Else
'sleep for specified time
Thread.Sleep(iInterval * 1000)
Dts.Events.FireInformation(1, "DEBUG:", sFilename & " not found successfully. Sleeping for " & iInterval & "* 1000", "", 0, False)
End If
'increment counter
iCounter = iCounter + 1
Loop

How do I create a simple Octave distributable without installing Octave

The Octave documentation on this subject is both intimidating and sparse.
I did not know where else to document the solution I found, so I am posting here. I apologize if that's inappropriate, but I want to help the next guy.
The following solution is for a simple windows distributable.
Use Case:
A solution is developed in Octave 3.2.4, and needs to be distributed to end-users with few computer skills. Installing and explaining Octave is impossible, the solution must be "one-click" or "brain-dead-simple."
Known Issues:
imread fails in 3.2.4 because file_in_path.m is wrong. You will need to update the file file_in_path.m to the following (just replace it):
function name=file_in_path(p,file)
idx=[1 findstr(p,pathsep) length(p)+1];
for i=1:length(idx)-1
if idx(i+1)-idx(i)<=1
dir=strcat(pwd,"/");
else
dir=p(idx(i)+1:idx(i+1)-1);
end
name = fullfile(dir, file);
fid = fopen(name,"r");
if fid >= 0
fclose(fid);
return
end
end
fid = fopen(file,"r");
if fid >= 0,
fclose(fid);
name=file;
return
end
name=[];
Solution: Create a distributable exe using mkoctfile, and package this exe with the core Octave files, and other .oct and .m files as necessary.
Step 1: Create a stand-alone executable.
You can see code that works here:
http://www.gnu.org/software/octave/doc/interpreter/Standalone-Programs.html
Particularly the file "embedded.cc".
I have simplified that file as follows:
#include <iostream>
#include <octave/oct.h>
#include <octave/octave.h>
#include <octave/parse.h>
int
main (int argc, char *argvc[])
{
string_vector argv (2);
argv(0) = "embedded";
argv(1) = "-q";
octave_main (2, argv.c_str_vec(), 1);
octave_value_list in = octave_value (argvc[1]);
octave_value_list out = feval ("your_custom_m_file", in);
if (!error_state && out.length () > 0)
{
}
else
{
std::cout << "invalid\n";
}
return 0;
}
Build this file with the command
mkoctfile --link-stand-alone embedded.cc -o embedded
It may throw warnings, but as long as it throws no errors, you should be fine. The file embedded.exe will be built, and can be run. The only issue is that it will lack all the goodies that make octave awesome. You will have to provide those.
Step 2: Create a distribution folder
You will need to create a copy of many of the Octave files. I suggest a directory specifically for this. At a minimum, you will need a copy of all or most of the DLLs in \bin. Additionally, place your distributable executable in this directory.
Step 3: Other files whack-a-mole
You will now need to find out what other files will be necessary to run your .m script. You can simplify this step by copying \oct\i686-pc-mingw32*.oct and \share\octave\3.2.4\m\*\*.m to the distribution directory, although this will be overkill, and will not actually prevent the whack-a-mole step.
Now, you must play whack-a-mole or the time-honored tradition of "where my includes be at, yo?"
Open a cmd prompt and navigate to your distribution folder.
Get rid of any useful PATH strings. Your customers won't have them.
Attempt to run the program embedded.exe. You will get an error such as the following:
embedded.exe
error: `max' undefined near line 83 column 22
error: evaluating argument list element number 1
error: evaluating argument list element number 1
error: called from:
error: T:\sms\Development\research\c2\disttest\strcat.m at line 83, column 3
error: T:\sms\Development\research\c2\disttest\file_in_path.m at line 5, column 10
error: T:\sms\Development\research\c2\disttest\imread.m at line 50, column 6
A Search in your Octave installation for "max". It will either be a .oct or a .m file. In this case, it is a .oct file, max.oct. Copy it to your distribution directory.
B You search for something obvious like "min", and get no results. This is because the Loadable Function "min" is in the .oct file "max.oct". Make a copy of max.oct, and rename it to min.oct. It will work now. How do you know where the functions are? I'm not sure. Most of them are in obvious places like "max.oct" for min, and "fft2.oct" for "ifft2.oct". Good luck with all that.
Repeat until your executable runs.
Just to add that if you want to run a script instead of an m function, then the line of the embedded.cc:
octave_value_list out = feval ("your_custom_m_file", in);
should be:
octave_value_list out = feval ("your_custom_m_script");
Also use 'which' to find where the missing functions are packed. For example for the min function:
octave:22> which min
min is a function from the file C:\Octave\Octave3.6.2_gcc4.6.2\lib\octave\3.6.2\oct\i686-pc-mingw32\max.oct
Something I found when linking my custom m file into an Octave standalone:
Needed #include <octave/toplev.h>
Replace return 0; (as above) with clean_up_and_exit(0);
Without these steps my program repeatedly crashed on exit.
Run mkoctfile --link-stand-alone embedded.cc -o embedded
from the octave solution and not from a batch file.
Just saved you half day (-;
In the above solution in bullet 4 B:
B You search for something obvious like "min", and get no results.
This is because the Loadable Function "min" is in the .oct file
"max.oct". Make a copy of max.oct, and rename it to min.oct. It will
work now.
This might not work if some function is being called from #folder function.m and also to avoid unnecessary duplicated files, just add the following code somewhere in your m file outside #folder
autoload ("min", "max.oct");
Likewise, it can be removed via
autoload ("min", "max.oct", "remove");
Ensure that the path to max.oct is provided here.
The above understanding is based on a file PKG_ADD and PKG_DEL in the communications package located at \Octave-4.0.1\lib\octave\packages\communications-1.2.1\i686-w64-mingw32-api-v50+\
Check out Stratego Octave Compiler.
(I've not tested it yet, but plan to do so in the next few days.)
I had that very same requirement (one-click, brain-dead-simple), so I made a setup that contained only curl.exe, the batch file below, an exe which was a .bat in disguise (simply calling the batch file below) and the .vbs script below (not writen by me). And of course my m-file.
This will download Octave 4.2.1 as a portable program (32 bit, otherwise we'dd have to download again if the system turns out to be 32 bit), unpack using the vbs script, move the contents to the same folder as the batch file and run it in GUI mode. Every next time the same script is called, it will only check if octave.bat is still there.
Of course this results in a huge waste of disk space, downloading the 280MB zip, which unpacks to over 1GB (which I make even worse by not deleting the zip afterwards), and you're stuck with a cmd window that is not easy to hide.
But it does offer the simplest solution I could find. It is also less likely to break in the future (either with an update of your own, or an update from Octave). Some glorious day, mkoktfile will actually be easy to use and will solve dependencies on its own, but until that day this remains the least headache-inducing solution I could find. And aspirins are more expensive than someone else's disk space.
::this file will test if the octave portable is downloaded and unpacked
#ECHO OFF
SET my_m_file=your_mfile.m
SET name_of_this_script=run_me.bat
::if the file exists, skip to the actual running.
IF EXIST "octave.bat" goto OctaveIsExtracted
IF EXIST "octave-4.2.1-w32.zip" goto OctaveIsDownloaded
ECHO The runtime (Octave portable 4.2.1) will now be downloaded.
ECHO This may take a long time, as it is about 280MB.
ECHO .
ECHO If this download restarts multiple times, you can manually download the octave-4.2.1-w32.zip from the GNU website. Make sure to unpack the contents.
::if this errors, you can uncomment the line with archive.org (which doesn't report total size during download)
curl http://ftp.gnu.org/gnu/octave/windows/octave-4.2.1-w32.zip > octave-4.2.1-w32.zip
::curl http://web.archive.org/web/20170827205614/https://ftp.gnu.org/gnu/octave/windows/octave-4.2.1-w32.zip > octave-4.2.1-w32.zip
:OctaveIsDownloaded
::check to see if the file size is the correct size to assume a successful download
::if the file size is incorrect, delete the file, restart this script to attempt a new download
::file size should be 293570269 bytes
call :filesize octave-4.2.1-w32.zip
IF /I "%size%" GEQ "293560000" goto OctaveIsDownloadedSuccessfully
del octave-4.2.1-w32.zip
::start new instance and exit and release this one
start %name_of_this_script%
exit
:OctaveIsDownloadedSuccessfully
IF EXIST "octave.bat" goto OctaveIsExtracted
::unzip and move those contents to the current folder
ECHO Unzipping octave portable, this may take a moment.
cscript //B j_unzip.vbs octave-4.2.1-w32.zip
SET src_folder=octave-4.2.1
SET tar_folder=%cd%
for /f %%a IN ('dir "%src_folder%" /b') do move %src_folder%\%%a %tar_folder%
pause
:OctaveIsExtracted
octave.bat %my_m_file%
goto :eof
:filesize
set size=%~z1
exit /b 0
And j_unzip.vbs
' j_unzip.vbs
'
' UnZip a file script
'
' By Justin Godden 2010
'
' It's a mess, I know!!!
'
' Dim ArgObj, var1, var2
Set ArgObj = WScript.Arguments
If (Wscript.Arguments.Count > 0) Then
var1 = ArgObj(0)
Else
var1 = ""
End if
If var1 = "" then
strFileZIP = "example.zip"
Else
strFileZIP = var1
End if
'The location of the zip file.
REM Set WshShell = CreateObject("Wscript.Shell")
REM CurDir = WshShell.ExpandEnvironmentStrings("%%cd%%")
Dim sCurPath
sCurPath = CreateObject("Scripting.FileSystemObject").GetAbsolutePathName(".")
strZipFile = sCurPath & "\" & strFileZIP
'The folder the contents should be extracted to.
outFolder = sCurPath
'original line: outFolder = sCurPath & "\"
WScript.Echo ( "Extracting file " & strFileZIP)
Set objShell = CreateObject( "Shell.Application" )
Set objSource = objShell.NameSpace(strZipFile).Items()
Set objTarget = objShell.NameSpace(outFolder)
intOptions = 256
objTarget.CopyHere objSource, intOptions
WScript.Echo ( "Extracted." )