I'm writing a PowerShell script that runs a couple of background jobs. Some of these background jobs will use the same set of constants or utility functions, like so:
$FirstConstant = "Not changing"
$SecondConstant = "Also not changing"
function Do-TheThing($thing)
{
# Stuff
}
$FirstJob = Start-Job -ScriptBlock {
Do-TheThing $using:FirstConstant
}
$SecondJob = Start-Job -ScriptBlock {
Do-TheThing $using:FirstConstant
Do-TheThing $using:SecondConstant
}
If I wanted to share variables (or, in this case, constants) in child scopes, I'd prefix the variable references with $using:. I can't do that with functions, though; running this code as-is returns an error:
The term 'Do-TheThing' is not recognized as the name of a cmdlet, function, script file, or operable program.
My question is this: How can my background jobs use a small utility function that I've defined in a higher scope?
If the function in the higher scope is in the same (non-)module scope in the same session, your code implicitly sees it, due to PowerShell's dynamic scoping.
However, background jobs run in a separate process (child process), so anything from the caller's scope must be passed explicitly to this separate session.
This is trivial for variable values, with the $using: scope, but less obvious for functions, but it can be made to work with a bit of duplication, by passing a function's body via namespace variable notation:
# The function to call from the background job.
Function Do-TheThing { param($thing) "thing is: $thing" }
$firstConstant = 'Not changing'
Start-Job {
# Define function Do-TheThing here in the background job, using
# the caller's function *body*.
${function:Do-TheThing} = ${using:function:Do-TheThing}
# Now call it, with a variable value from the caller's scope
Do-TheThing $using:firstConstant
} | Receive-Job -Wait -AutoRemoveJob
The above outputs 'thing is: Not changing', as expected.
Related
The documentation probably explains it very well but I do not see the difference between this 2 commands in my case :
method dir {} {
puts "method dir..."
}
method pseudomethod {} {
set vardir [my dir]
set vardir [[self] dir]
}
The only difference I can see is that with [self] I can pass it as an argument in a procedure and not with my.
What is the best solution in my case ?
Both solutions have equal performance ?
The self command (with no extra arguments) is equivalent to self object which returns the current public name of the object that is executing the method (you can rename the object). The self command overall provides access to bits of “runtime” state.
The my command is actually the object's internal name; it's created in each object's instance namespace. You can invoke all exported and non-exported methods via my, unlike with the public name. This makes it useful for both calling your internal methods directly, and also for setting up things like callbacks to internal methods (you'll need something like namespace which or namespace code when setting up the callback).
Unlike with the public name, you can delete the internal name command without automatically destroying the object. It'll likely break code (your methods most probably) if you do that, but the base system allows you to do it.
Aside: Tcl 8.7 includes this helper procedure (which also works in 8.6) for creating callback scripts within methods (the funny name means it gets mapped into your methods automatically as callback):
proc ::oo::Helpers::callback {method args} {
list [uplevel 1 {::namespace which my}] $method {*}$args
}
In this case, if the callback was exported, you'd be able to do this instead:
proc ::oo::Helpers::callback {method args} {
list [uplevel 1 self] $method {*}$args
}
but that would be more vulnerable to rename problems. (In all cases, the uplevel 1 is because we want to run a little bit of name-resolving code in the calling context, not inside the scope of the procedure itself.)
I'm not sure how they are implemented, but one reason you'd want to use my is to access non-exported (private) methods. A demo:
oo::class create Foo {
method PrivateMethod {} {puts "this is PrivateMethod"}
method publicMethod {} {puts "this is publicMethod"}
method test {} {
my publicMethod
my PrivateMethod
[self] publicMethod
[self] PrivateMethod
}
}
then:
% Foo create foo
::foo
% foo test
this is publicMethod
this is PrivateMethod
this is publicMethod
unknown method "PrivateMethod": must be destroy, publicMethod or test
my is the mechanism for an object to invoke its methods.
self is the mechanism for introspection on how the current method was called.
Spend some time with the my and self man pages.
Consider the following arbitrary function and test cases:
Function Foo-MyBar {
Param(
[Parameter(Mandatory=$false)]
[ScriptBlock] $Filter
)
if (!$Filter) {
$Filter = { $true }
}
#$Filter = $Filter.GetNewClosure()
Get-ChildItem "$env:SYSTEMROOT" | Where-Object $Filter
}
##################################
$private:pattern = 'T*'
Get-Help Foo-MyBar -Detailed
Write-Host "`n`nUnfiltered..."
Foo-MyBar
Write-Host "`n`nTest 1:. Piped through Where-Object..."
Foo-MyBar | Where-Object { $_.Name -ilike $private:pattern }
Write-Host "`n`nTest 2:. Supplied a naiive -Filter parameter"
Foo-MyBar -Filter { $_.Name -ilike $private:pattern }
In Test 1, we pipe the results of Foo-MyBar through a Where-Object filter, which compares the objects returned to a pattern contained in a private-scoped variable $private:pattern. In this case, this correctly returns all the files/folders in C:\ which start with the letter T.
In Test 2, we pass the same filtering script directly as a parameter to Foo-MyBar. However, by the time Foo-MyBar gets to running the filter, $private:pattern is not in scope, and so this returns no items.
I understand why this is the case -- because the ScriptBlock passed to Foo-MyBar is not a closure, so does not close over the $private:pattern variable and that variable is lost.
I note from comments that I previously had a flawed third test, which tried to pass {...}.GetNewClosure(), but this does not close over private-scoped variables -- thanks #PetSerAl for helping me clarify that.
The question is, how does Where-Object capture the value of $private:pattern in Test 1, and how do we achieve the same behaviour in our own functions/cmdlets?
(Preferably without requiring the caller to have to know about closures, or know to pass their filter script as a closure.)
I note that, if I uncomment the $Filter = $Filter.GetNewClosure() line inside Foo-MyBar, then it never returns any results, because $private:pattern is lost.
(As I said at the top, the function and parameter are arbitrary here, as a shortest-form reproduction of my real problem!)
The example given does not work because calling a function will enter a new scope by default. Where-Object will still invoke the filter script without entering one, but the scope of the function does not have the private variable.
There's three ways around this.
Put the function in a different module than the caller
Every module has a SessionState which has its own stack of SessionStateScopes. Every ScriptBlock is tied to the SessionState is was parsed in.
If you call a function defined in a module, a new scope is created within that module's SessionState, but not within the top level SessionState. Therefore when Where-Object invokes the filter script without entering a new scope, it does so on the current scope for the SessionState to which that ScriptBlock is tied.
This is a bit fragile, because if you want to call that function from your module, well you can't. It'll have the same issue.
Call the function with the dot source operator
You most likely already know the dot-source operator (.) for invoking script files without creating a new scope. That also works with command names and ScriptBlock objects.
. { 'same scope' }
. Foo-MyBar
Note, however, that this will invoke the function within the current scope of the SessionState that the function is from, so you cannot rely on . to always execute in the caller's current scope. Therefore, if you invoke functions associated with a different SessionState with the dot-source operator - such as functions defined in a (different) module - it may have unintended effects. Variables created will persist to future function invocations and any helper functions defined within the function itself will also persist.
Write a Cmdlet
Compiled commands (cmdlets) do not create a new scope when invoked. You can also use similar API's to what Where-Object use (though not the exact same ones)
Here's a rough implementation of how you could implement Where-Object using public API's
using System.Management.Automation;
namespace MyModule
{
[Cmdlet(VerbsLifecycle.Invoke, "FooMyBar")]
public class InvokeFooMyBarCommand : PSCmdlet
{
[Parameter(ValueFromPipeline = true)]
public PSObject InputObject { get; set; }
[Parameter(Position = 0)]
public ScriptBlock FilterScript { get; set; }
protected override void ProcessRecord()
{
var filterResult = InvokeCommand.InvokeScript(
useLocalScope: false,
scriptBlock: FilterScript,
input: null,
args: new[] { InputObject });
if (LanguagePrimitives.IsTrue(filterResult))
{
WriteObject(filterResult, enumerateCollection: true);
}
}
}
}
how does Where-Object capture the value of $private:pattern in Test 1
As can be seen in the source code for Where-Object in PowerShell Core, PowerShell internally invokes the filter script without confining it to its own local scope (_script is the private backing field for the FilterScript parameter, notice the useLocalScope: false argument passed to DoInvokeReturnAsIs()):
protected override void ProcessRecord()
{
if (_inputObject == AutomationNull.Value)
return;
if (_script != null)
{
object result = _script.DoInvokeReturnAsIs(
useLocalScope: false, // <-- notice this named argument right here
errorHandlingBehavior: ScriptBlock.ErrorHandlingBehavior.WriteToCurrentErrorPipe,
dollarUnder: InputObject,
input: new object[] { _inputObject },
scriptThis: AutomationNull.Value,
args: Utils.EmptyArray<object>());
if (_toBoolSite.Target.Invoke(_toBoolSite, result))
{
WriteObject(InputObject);
}
}
// ...
}
how do we achieve the same behaviour in our own functions/cmdlets?
We don't - DoInvokeReturnAsIs() (and similar scriptblock invocation facilities) are marked internal and can therefore only be invoked by types contained in the System.Management.Automation assembly
With this Tcl script: A.tcl
itcl::class ::A {
variable list1 {}
public method getList {} {
return $list1
}
}
I do this:
Start the tcl shell and interactively do source A.tcl
then make changes to getList method in A.tcl
To make the changes effective, I do re-source the file A.tcl
When I re-source, I get the following error
% source /home/manid/scripts/test.tcl
class "A" already exists
How can i overcome this error? Is there a way to get the latest changes in the class definition without exiting the shell?
You need to write your code somewhat differently. In particular, you have to put the definitions of the body of the methods (which can be repeated) outside the declaration of the class (which can't be repeated). Then, you do a conditional class creation (with itcl::is class as the tester) and use itcl::body to actually provide the method definitions.
According to these principles, rewriting your A.tcl to be:
if {![itcl::is class ::A]} {
itcl::class ::A {
variable list1 {}
# *Dummy* method body; method must exist, but may be empty
public method getList {} {}
}
}
itcl::body ::A::getList {} {
return $list1
}
would allow you to source that multiple times to change the method bodies however you wanted. This doesn't give you freedom to change everything (e.g., the variable declarations and scope rules are fixed); you need to switch to something like TclOO to get that sort of flexibility.
I have this powershell script:
function Func1 ($val)
{
Write-Host "$val is processed by Func1";
}
function Func2($val)
{
Invoke-Command -ScriptBlock `
${function:Func1} -ArgumentList "$val is processed by Func2 and";
}
function Func3($val)
{
$function:Func2.Invoke("$val is processed by Func3 and");
}
Func3 "Value";
This works - it outputs Value is processed by Func3 and is processed by Func2 and is processed by Func1 - but I am confused at two things:
What does the ${function:function-name} code (i.e. a dollar sign followed by an opening curly brace followed by function followed by a colon followed by the name of the function followed by a closing curly brace) in Func2 mean? I can see that it invokes Func1, but I don't really understand why it works.
What does the $function:function-name.Invoke code in Func3 mean? I sense that it is using script block functionality, since the Invoke method is called, but it's not clear to me how $function.function-name is a script block.
function: is the PsDrive for the Function provider. All functions are stored on this drive. There are other PsDrives including variable: and env:. Check out Get-PsProvider and Get-PsDrive for more.
To access a function from the function: drive (get its contents, not call it), use $function:foo where foo is the name of the function in which to access.
Curly braces are only required when you are accessing a variable that has special character in its name.
The contents of functions are script blocks, which is why it's being used as the scriptblock parameter for Invoke-Command.
Every thing in the function: psdrive will be a script block, and scriptblock objects have an Invoke method which allows you to execute them.
I'm using a function which creates some variables I'd like to use after the function has processed.
I've tried accessing them directly but I can't. How would I go about doing this?
Variables inside functions don't survive after the function has been run. If you want to access them after the function has processed, prefix them with a scope modifier.
PS> function test-var{ $script:var='foo' }
PS> test-var # excute the function
PS> $var #print var
foo
Type this in your console for more information:
PS> Get-Help about_Scopes
as Shay pointed out you can create what are called global variables inside the function scope that will be available at higher level scopes. However global variables are generally not a good idea and I'd like to suggest some alternatives for you.
This is from the wikipedia global variable page:
They are usually considered bad practice precisely because of their
non-locality: a global variable can potentially be modified from
anywhere (unless they reside in protected memory or are otherwise
rendered read-only), and any part of the program may depend on it.[1]
A global variable therefore has an unlimited potential for creating
mutual dependencies, and adding mutual dependencies increases
complexity.
Some alternatives:
Make the function return data needed by the caller. Powershell functions should generally return the data related to the noun in the Powershell function naming convention of Verb-Noun. If you need to return other data not associated to the noun consider making a second function.
function Get-Directories {
param ([string] $Path)
# Code to get or create objects here.
$dirs = Get-ChildItem -Path $Path | where {$_.PsIsContainer}
# Explicitly return data to the caller.
return $dirs
}
$myDirs = Get-Directories -Path 'C:\'
Use a reference variable. A reference passes a variable's address in memory to a function. When the function changes the variable's data it will be accessible outside of the function but the variable's scope will not of been changed.
function Get-Directories {
param ([string] $Path, [ref] $Directories)
$Directories.Value = Get-ChildItem -Path $Path | where {$_.PsIsContainer}
}
$myDirs = $null
Get-Directories -Path 'C:\' -Directories ([ref] $myDirs)
Hope this helps. Happy coding :-)
If you run your function with . , the function will execute in your scope, and all of the variables defined within the function will be available to the caller.
i.e.
. $func
$myPrivateVariable # Now Set in the parent scope.
If the function is in a module, you can also use this sort of trick to access the module's scope:
$m =Get-Module myModule
. $m { $myPrivateModuleVariable }
Hope this Helps