To control for a minority positive class for the project I'm working on, I'm implementing step_downsample() in my recipe. I'm also using 10-fold cross-validation to mitigate bias. When I use a workflow to wrap up the learner, recipe, a grid search, and the CV folds, does the workflow apply the recipe steps to each individual fold prior to model training? The order of operations is hazy to me and I wasn't able to find any satisfactory answers in the documentation. Thanks!
I think you might find this chapter helpful, especially the section "Where does the model begin and end?".
Yes, in tidymodels, the preprocessing recipe (i.e. feature engineering procedure) is considered part of the modeling process and is trained on each fold like the learner.
You can see this happening in the logging if you set verbose = TRUE during tuning:
library(tidymodels)
library(themis)
#>
#> Attaching package: 'themis'
#> The following objects are masked from 'package:recipes':
#>
#> step_downsample, step_upsample
data(Ionosphere, package = "mlbench")
svm_mod <-
svm_rbf(cost = tune(), rbf_sigma = tune()) %>%
set_mode("classification") %>%
set_engine("kernlab")
iono_rec <-
recipe(Class ~ ., data = Ionosphere) %>%
# remove any zero variance predictors
step_zv(all_predictors()) %>%
# remove any linear combinations
step_lincomb(all_numeric()) %>%
step_downsample(Class)
set.seed(123)
iono_rs <- bootstraps(Ionosphere, times = 5)
set.seed(325)
svm_mod %>%
tune_grid(
iono_rec,
resamples = iono_rs,
control = control_grid(verbose = TRUE)
)
#> i Bootstrap1: preprocessor 1/1
#> ✓ Bootstrap1: preprocessor 1/1
#> i Bootstrap1: preprocessor 1/1, model 1/10
#> ✓ Bootstrap1: preprocessor 1/1, model 1/10
#> i Bootstrap1: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 2/10
#> ✓ Bootstrap1: preprocessor 1/1, model 2/10
#> i Bootstrap1: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 3/10
#> ✓ Bootstrap1: preprocessor 1/1, model 3/10
#> i Bootstrap1: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 4/10
#> ✓ Bootstrap1: preprocessor 1/1, model 4/10
#> i Bootstrap1: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 5/10
#> ✓ Bootstrap1: preprocessor 1/1, model 5/10
#> i Bootstrap1: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 6/10
#> ✓ Bootstrap1: preprocessor 1/1, model 6/10
#> i Bootstrap1: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 7/10
#> ✓ Bootstrap1: preprocessor 1/1, model 7/10
#> i Bootstrap1: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 8/10
#> ✓ Bootstrap1: preprocessor 1/1, model 8/10
#> i Bootstrap1: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 9/10
#> ✓ Bootstrap1: preprocessor 1/1, model 9/10
#> i Bootstrap1: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap1: preprocessor 1/1, model 10/10
#> ✓ Bootstrap1: preprocessor 1/1, model 10/10
#> i Bootstrap1: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap2: preprocessor 1/1
#> ✓ Bootstrap2: preprocessor 1/1
#> i Bootstrap2: preprocessor 1/1, model 1/10
#> ✓ Bootstrap2: preprocessor 1/1, model 1/10
#> i Bootstrap2: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 2/10
#> ✓ Bootstrap2: preprocessor 1/1, model 2/10
#> i Bootstrap2: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 3/10
#> ✓ Bootstrap2: preprocessor 1/1, model 3/10
#> i Bootstrap2: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 4/10
#> ✓ Bootstrap2: preprocessor 1/1, model 4/10
#> i Bootstrap2: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 5/10
#> ✓ Bootstrap2: preprocessor 1/1, model 5/10
#> i Bootstrap2: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 6/10
#> ✓ Bootstrap2: preprocessor 1/1, model 6/10
#> i Bootstrap2: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 7/10
#> ✓ Bootstrap2: preprocessor 1/1, model 7/10
#> i Bootstrap2: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 8/10
#> ✓ Bootstrap2: preprocessor 1/1, model 8/10
#> i Bootstrap2: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 9/10
#> ✓ Bootstrap2: preprocessor 1/1, model 9/10
#> i Bootstrap2: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap2: preprocessor 1/1, model 10/10
#> ✓ Bootstrap2: preprocessor 1/1, model 10/10
#> i Bootstrap2: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap3: preprocessor 1/1
#> ✓ Bootstrap3: preprocessor 1/1
#> i Bootstrap3: preprocessor 1/1, model 1/10
#> ✓ Bootstrap3: preprocessor 1/1, model 1/10
#> i Bootstrap3: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 2/10
#> ✓ Bootstrap3: preprocessor 1/1, model 2/10
#> i Bootstrap3: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 3/10
#> ✓ Bootstrap3: preprocessor 1/1, model 3/10
#> i Bootstrap3: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 4/10
#> ✓ Bootstrap3: preprocessor 1/1, model 4/10
#> i Bootstrap3: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 5/10
#> ✓ Bootstrap3: preprocessor 1/1, model 5/10
#> i Bootstrap3: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 6/10
#> ✓ Bootstrap3: preprocessor 1/1, model 6/10
#> i Bootstrap3: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 7/10
#> ✓ Bootstrap3: preprocessor 1/1, model 7/10
#> i Bootstrap3: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 8/10
#> ✓ Bootstrap3: preprocessor 1/1, model 8/10
#> i Bootstrap3: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 9/10
#> ✓ Bootstrap3: preprocessor 1/1, model 9/10
#> i Bootstrap3: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap3: preprocessor 1/1, model 10/10
#> ✓ Bootstrap3: preprocessor 1/1, model 10/10
#> i Bootstrap3: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap4: preprocessor 1/1
#> ✓ Bootstrap4: preprocessor 1/1
#> i Bootstrap4: preprocessor 1/1, model 1/10
#> ✓ Bootstrap4: preprocessor 1/1, model 1/10
#> i Bootstrap4: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 2/10
#> ✓ Bootstrap4: preprocessor 1/1, model 2/10
#> i Bootstrap4: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 3/10
#> ✓ Bootstrap4: preprocessor 1/1, model 3/10
#> i Bootstrap4: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 4/10
#> ✓ Bootstrap4: preprocessor 1/1, model 4/10
#> i Bootstrap4: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 5/10
#> ✓ Bootstrap4: preprocessor 1/1, model 5/10
#> i Bootstrap4: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 6/10
#> ✓ Bootstrap4: preprocessor 1/1, model 6/10
#> i Bootstrap4: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 7/10
#> ✓ Bootstrap4: preprocessor 1/1, model 7/10
#> i Bootstrap4: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 8/10
#> ✓ Bootstrap4: preprocessor 1/1, model 8/10
#> i Bootstrap4: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 9/10
#> ✓ Bootstrap4: preprocessor 1/1, model 9/10
#> i Bootstrap4: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap4: preprocessor 1/1, model 10/10
#> ✓ Bootstrap4: preprocessor 1/1, model 10/10
#> i Bootstrap4: preprocessor 1/1, model 10/10 (predictions)
#> i Bootstrap5: preprocessor 1/1
#> ✓ Bootstrap5: preprocessor 1/1
#> i Bootstrap5: preprocessor 1/1, model 1/10
#> ✓ Bootstrap5: preprocessor 1/1, model 1/10
#> i Bootstrap5: preprocessor 1/1, model 1/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 2/10
#> ✓ Bootstrap5: preprocessor 1/1, model 2/10
#> i Bootstrap5: preprocessor 1/1, model 2/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 3/10
#> ✓ Bootstrap5: preprocessor 1/1, model 3/10
#> i Bootstrap5: preprocessor 1/1, model 3/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 4/10
#> ✓ Bootstrap5: preprocessor 1/1, model 4/10
#> i Bootstrap5: preprocessor 1/1, model 4/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 5/10
#> ✓ Bootstrap5: preprocessor 1/1, model 5/10
#> i Bootstrap5: preprocessor 1/1, model 5/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 6/10
#> ✓ Bootstrap5: preprocessor 1/1, model 6/10
#> i Bootstrap5: preprocessor 1/1, model 6/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 7/10
#> ✓ Bootstrap5: preprocessor 1/1, model 7/10
#> i Bootstrap5: preprocessor 1/1, model 7/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 8/10
#> ✓ Bootstrap5: preprocessor 1/1, model 8/10
#> i Bootstrap5: preprocessor 1/1, model 8/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 9/10
#> ✓ Bootstrap5: preprocessor 1/1, model 9/10
#> i Bootstrap5: preprocessor 1/1, model 9/10 (predictions)
#> i Bootstrap5: preprocessor 1/1, model 10/10
#> ✓ Bootstrap5: preprocessor 1/1, model 10/10
#> i Bootstrap5: preprocessor 1/1, model 10/10 (predictions)
#> # Tuning results
#> # Bootstrap sampling
#> # A tibble: 5 x 4
#> splits id .metrics .notes
#> <list> <chr> <list> <list>
#> 1 <split [351/136]> Bootstrap1 <tibble [20 × 6]> <tibble [0 × 1]>
#> 2 <split [351/124]> Bootstrap2 <tibble [20 × 6]> <tibble [0 × 1]>
#> 3 <split [351/134]> Bootstrap3 <tibble [20 × 6]> <tibble [0 × 1]>
#> 4 <split [351/130]> Bootstrap4 <tibble [20 × 6]> <tibble [0 × 1]>
#> 5 <split [351/138]> Bootstrap5 <tibble [20 × 6]> <tibble [0 × 1]>
Created on 2021-03-10 by the reprex package (v1.0.0)
In this particular case, it walks through each resample and first trains the recipe, then fits the first learner/parameter option, then evaluates predictions on the heldout set for that resample and learner/parameter option, then goes to the next learner option.
We have a function, called via the input paste reactor to process some text in a drawing. We are getting different results using Autodesk 2021 from a script migrated from 2019 and I can't figure out why there is a difference. The script is below:
;; GDD parser to remove unnecessary info from the paste to prepare for processing
;; Modified Lee mac's unformat function - removes formatting from a string
(defun GDD:removeinfo ( rgx str )
(if
(null
(vl-catch-all-error-p
(setq str
(apply
'(lambda nil
(vlax-put-property rgx 'global actrue)
(vlax-put-property rgx 'multiline actrue)
(vlax-put-property rgx 'ignorecase acfalse)
(foreach pair
'(
("\\032" . "\\\\\\\\")
("\n" . "\\\\P")
("$1" . "\\\\(\\\\[ACcFfHKkLlOopQTW])|\\\\[ACcFfHKkLlOopQTW][^\\\\;]*;|\\\\;[ACcFfKkHLlOopQTW]")
("$1$2/$3" . "([^\\\\])\\\\S([^;]*)[/#\\^]([^;]*);")
("$1$2" . "\\\\(\\\\S)|[\\\\](})|}")
("$1" . "[\\\\]({)|{")
("\\$1$2$3" . "(\\\\[ACcFfHKkLlOoPpQSTW])|({)|(})")
("\\\\" . "\032")
;; Added from LMs original quickunformat
("" . "(?:.*\\n)*Const\\s+.*\\n")
("" . "\\w\\w\\d?\\s+\\d+\\s\\d+-\\d+-\\d+")
("" . "^\\s+\\n")
)
(vlax-put-property rgx 'pattern (cdr pair))
(setq str (vlax-invoke rgx 'replace str (car pair)))
)
) nil
)
)
)
)
str
(prompt (strcat "\nError: " (vl-catch-all-error-message str)))
)
)
The input string from the paste is:
Ranford Rd, Perth WA, Australia
Clear Markers
-3773641.56683170, 12902093.08140300 | 0, 01 feature(s) selected on 1 layer(s)1:
2256.9944
979.35 x 559.54 m
Attribute Value
SUBTYPE ROUTE
ATTRIBUTE
TLS_ID 15006025041
Date 30-06-1997
Length 231.0
Shared NO
Const MULTI-CONDUIT
AA P100 312F - SMOF FNPEHJC/STDCNVL F CNVL 4701:CNVL AD-CNVL AE/1-312
312F - SMOF FNPEHJ/STD CNVL F HILN 4602:CNVL AE-CNVL BW/1-312
120F - SMOF FNPEHJ/STD CNVL F BATA 1001:CNVL AE-CNVL CR/1-120
24F - SMOF FNPEHJ CNVL F CNVL 1001:CNVL AD-CNVL AE/1-24
12F - SMOF FNPEHJC/STDCNVL F BATA 1001:CNVL DL-CNVL HX/1-12
AB P100 200 0.40 CPFUT MB CNVL CA1:B1901-2100
AA 15038127628 30-06-1997
AB 15038127629 30-06-1997
The output (for str) in the function should be:
AA P100 312F - SMOF FNPEHJC/STDCNVL F CNVL 4701:CNVL AD-CNVL AE/1-312
312F - SMOF FNPEHJ/STD CNVL F HILN 4602:CNVL AE-CNVL BW/1-312
120F - SMOF FNPEHJ/STD CNVL F BATA 1001:CNVL AE-CNVL CR/1-120
24F - SMOF FNPEHJ CNVL F CNVL 1001:CNVL AD-CNVL AE/1-24
12F - SMOF FNPEHJC/STDCNVL F BATA 1001:CNVL DL-CNVL HX/1-12
AB P100 200 0.40 CPFUT MB CNVL CA1:B1901-2100
However when we process this function with the same input in Autodesk 2021 the output (for str) is:
Ranford Rd, Perth WA, Australia
It looks like the function errors or returns on the first line break it receives and exits, although in the debugger I can see the regex pairs being processed. I'm lost as to why there is any difference in the new version? Many thanks for your help.
This is caused by the new AutoLISP Unicode character support introduced in AutoCAD 2021, alongside the introduction of VS Code as the primary AutoLISP Editor.
To revert to behaviour exhibited by AutoCAD 2020 and earlier, you can set the new LISPSYS system variable to 0:
LISPSYS (System Variable)
Controls the default AutoLISP development environment and the editor
launched with the VLISP command.
0
Visual LISP IDE (VLIDE) is set as the default editor, however
AutoLISP functions don't fully support Unicode characters. AutoLISP source (LSP) files when saved and compiled use the ASCII (MBCS) character set. Note: This setting results in the behavior of AutoCAD 2020 and earlier releases, and is supported on Windows only.
1
Visual Studio (VS) Code is set as the default editor and AutoLISP
functions fully support Unicode characters. AutoLISP source (LSP) files, when saved, use the encoding set in VS Code, and when compiled, they use the Unicode character set.
2
Visual Studio (VS) Code is set as the default editor and AutoLISP
functions fully support Unicode characters. AutoLISP source (LSP) files, when saved, use the encoding set in VS Code, and when compiled they use the ASCII (MBCS) character set.
This is a bit of a strange question, bear with me: I stumbled upon some data in a machine-readable format in the source code of an HTML page (inside a comment above the opening <html> tag), but I have never seen data that looked anything like this format.
Can anyone identify this data format? Does anyone know if there is an established/documented standard for transmitting/storing data like this? (my hope is that, if this is a standard data format, I can find pre-existing libraries for parsing it and save myself from reinventing the wheel)
Here's the raw data (I omitted some of the data just to keep the post short):
<!--
Fin :: 0
ErrorMsg ::
MoreErrors ::
MFErrorArray :: ARRAY[2 * 120]
[1]
[0:ErrorCode]{ }
[1:ArrayIndex]{ }
MFErrorArray2 :: ARRAY[3 * 60]
[1]
[0:ErrorCode2]{ }
[1:Substitution]{ }
[2:ArrayIndex2]{ }
NotUsed ::
AllControlNumber ::
Datu ::
Pgm :: BXS2BL40
VlNumbHous ::
NmStrt ::
NmBoro ::
VlBin ::
VlNumbZip ::
VlTaxBlock ::
VlTaxLot ::
VlCensTract ::
VlHlthArea ::
HseLo ::
HseHi ::
GlJobType ::
GlPageN :: 0001
GlRecCountN :: 0000000517
FoilIndicator ::
GlMax ::
DebugMsg ::
VlLicnType :: B
NmLicnType :: ELECTRICAL FIRM
VlLicn :: ARRAY[13 * 70]
[1]
[0:NmLicn]{}
[1:VlNumbLIcn]{B001572}
[2:StLicn]{INACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A & A ELEC. CONTRACTING}
[5:NmBusn2]{}
[6:NbIsn]{0000023530}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
[2]
[0:NmLicn]{}
[1:VlNumbLIcn]{B002944}
[2:StLicn]{ACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A & A ELEC'L CONTR'G CORP}
[5:NmBusn2]{}
[6:NbIsn]{0000024858}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
[3]
[0:NmLicn]{}
[1:VlNumbLIcn]{B000014}
[2:StLicn]{INACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A & A ELECTRIC INC.}
[5:NmBusn2]{}
[6:NbIsn]{0000021979}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
*** I've removed entries 4 through 67 in this array for sake of brevity ***
[68]
[0:NmLicn]{}
[1:VlNumbLIcn]{B003051}
[2:StLicn]{ACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A.L. ELECTRICAL CORP.}
[5:NmBusn2]{}
[6:NbIsn]{0000024954}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
[69]
[0:NmLicn]{}
[1:VlNumbLIcn]{B002419}
[2:StLicn]{ACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A.M. ELECTRIC CORP. OF NY}
[5:NmBusn2]{}
[6:NbIsn]{0000024375}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
[70]
[0:NmLicn]{}
[1:VlNumbLIcn]{B003863}
[2:StLicn]{ACTIVE}
[3:DtLicnExp]{12312050}
[4:NmBusn1]{A.M.A HOLDINGS INC.D/B/A}
[5:NmBusn2]{}
[6:NbIsn]{0000028205}
[7:FirmIsn]{}
[8:FirmLicenseNumber]{}
[9:JobCount]{0000000000}
[10:LLicenseClass]{}
[11:LLicenseClassType]{}
[12:GreenFlag]{N}
-->
This data excerpt seems to contain information about drivers, vehicles, licensing, etc.
I have not personally seen data formatted exactly this way before, but if you came across it on a commercial website it is likely either a highly specialized data standard for that industry or some ad hoc solution thrown together by that company in lieu of a better standard. Perhaps if you could share a link to the site we could dig into it further, in context.
It looks fairly straightforward though, why not just write a parsing algorithm?
I want to execute a CUDA kernel in python using Numbapro API. I have this code:
import math
import numpy
from numbapro import jit, cuda, int32, float32
from matplotlib import pyplot
#cuda.jit('void(float32[:], float32[:], float32[:], float32[:], float32, float32, float32, int32)')
def calculate_velocity_field(X, Y, u_source, v_source, x_source, y_source, strength_source, N):
start = cuda.blockIdx.x * cuda.blockDim.x + cuda.threadIdx.x
end = N
stride = cuda.gridDim.x * cuda.blockDim.x
for i in range(start, end, stride):
u_source[i] = strength_source/(2*math.pi) * (X[i]-x_source)/((X[i]-x_source)**2 + (Y[i]-y_source)**2)
v_source[i] = strength_source/(2*math.pi) * (Y[i]-x_source)/((X[i]-x_source)**2 + (Y[i]-y_source)**2)
def main():
N = 200 # number of points in each direction
x_start, x_end = -4.0, 4.0 # boundaries in the x-direction
y_start, y_end = -2.0, 2.0 # boundaries in the y-direction
x = numpy.linspace(x_start, x_end, N) # creates a 1D-array with the x-coordinates
y = numpy.linspace(y_start, y_end, N) # creates a 1D-array with the y-coordinates
X, Y = numpy.meshgrid(x, y) # generates a mesh grid
strength_source = 5.0 # source strength
x_source, y_source = -1.0, 0.0 # location of the source
start = timer()
#calculate grid dimensions
blockSize = 1024
gridSize = int(math.ceil(float(N)/blockSize))
#transfer memory to device
X_d = cuda.to_device(X)
Y_d = cuda.to_device(Y)
u_source_d = cuda.device_array_like(X)
v_source_d = cuda.device_array_like(Y)
#launch kernel
calculate_velocity_field[gridSize,blockSize](X_d,Y_d,u_source_d,v_source_d,x_source,y_source,strength_source,N)
#transfer memory to host
u_source = numpy.empty_like(X)
v_source = numpy.empty_like(Y)
u_source_d.to_host(u_source)
v_source_d.to_host(v_source)
elapsed_time = timer() - start
print("Exec time with GPU %f s" % elapsed_time)
if __name__ == "__main__":
main()
Is giving me this error:
NvvmError Traceback (most recent call last)
<ipython-input-17-85e4a6e56a14> in <module>()
----> 1 #cuda.jit('void(float32[:], float32[:], float32[:], float32[:], float32, float32, float32, int32)')
2 def calculate_velocity_field(X, Y, u_source, v_source, x_source, y_source, strength_source, N):
3 start = cuda.blockIdx.x * cuda.blockDim.x + cuda.threadIdx.x
4 end = N
5 stride = cuda.gridDim.x * cuda.blockDim.x
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/decorators.py in kernel_jit(func)
89 # Force compilation for the current context
90 if bind:
---> 91 kernel.bind()
92
93 return kernel
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/compiler.py in bind(self)
319 Force binding to current CUDA context
320 """
--> 321 self._func.get()
322
323 #property
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/compiler.py in get(self)
254 cufunc = self.cache.get(device.id)
255 if cufunc is None:
--> 256 ptx = self.ptx.get()
257
258 # Link
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/compiler.py in get(self)
226 arch = nvvm.get_arch_option(*cc)
227 ptx = nvvm.llvm_to_ptx(self.llvmir, opt=3, arch=arch,
--> 228 **self._extra_options)
229 self.cache[cc] = ptx
230 if config.DUMP_ASSEMBLY:
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/cudadrv/nvvm.py in llvm_to_ptx(llvmir, **opts)
420 cu.add_module(llvmir.encode('utf8'))
421 cu.add_module(libdevice.get())
--> 422 ptx = cu.compile(**opts)
423 return ptx
424
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/cudadrv/nvvm.py in compile(self, **options)
211 for x in opts])
212 err = self.driver.nvvmCompileProgram(self._handle, len(opts), c_opts)
--> 213 self._try_error(err, 'Failed to compile\n')
214
215 # get result
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/cudadrv/nvvm.py in _try_error(self, err, msg)
229
230 def _try_error(self, err, msg):
--> 231 self.driver.check_error(err, "%s\n%s" % (msg, self.get_log()))
232
233 def get_log(self):
~/.anaconda3/lib/python3.4/site-packages/numba/cuda/cudadrv/nvvm.py in check_error(self, error, msg, exit)
118 sys.exit(1)
119 else:
--> 120 raise exc
121
122
NvvmError: Failed to compile
libnvvm : error: -arch=compute_52 is an unsupported option
NVVM_ERROR_INVALID_OPTION
I tried another numbapro examples and the same error ocurrs.
I don't know if it's a bug of Numbapro that doesn't support 5.2 compute capability or it's a problem of Nvidia NVVM... suggestions?
In theory it should be supported, but I don't know what is happening.
I'm using Linux with CUDA 7.0 and driver version 346.29
Finally I found a solution here
Solution 1:
conda update cudatoolkit
Fetching package metadata: ....
# All requested packages already installed.
# packages in environment at ~/.anaconda3:
#
cudatoolkit 6.0 p0
It looks like me updating the CUDA toolkit doesn't update to CUDA 7.0. A second solution can be done:
Solution 2
conda install -c numba cudatoolkit
Fetching package metadata: ......
Solving package specifications: .
Package plan for installation in environment ~/.anaconda3:
The following packages will be downloaded:
package | build
---------------------------|-----------------
cudatoolkit-7.0 | 1 190.8 MB
The following packages will be UPDATED:
cudatoolkit: 6.0-p0 --> 7.0-1
Proceed ([y]/n)? y
Before:
In [4]: check_cuda()
------------------------------libraries detection-------------------------------
Finding cublas
located at ~/.anaconda3/lib/libcublas.so.6.0.37
trying to open library... ok
Finding cusparse
located at ~/.anaconda3/lib/libcusparse.so.6.0.37
trying to open library... ok
Finding cufft
located at ~/.anaconda3/lib/libcufft.so.6.0.37
trying to open library... ok
Finding curand
located at ~/.anaconda3/lib/libcurand.so.6.0.37
trying to open library... ok
Finding nvvm
located at ~/.anaconda3/lib/libnvvm.so.2.0.0
trying to open library... ok
finding libdevice for compute_20... ok
finding libdevice for compute_30... ok
finding libdevice for compute_35... ok
-------------------------------hardware detection-------------------------------
Found 1 CUDA devices
id 0 b'GeForce GTX 970' [SUPPORTED]
compute capability: 5.2
pci device id: 0
pci bus id: 7
Summary:
1/1 devices are supported
PASSED
Out[4]: True
After:
In [6]: check_cuda()
------------------------------libraries detection-------------------------------
Finding cublas
located at ~/.anaconda3/lib/libcublas.so.7.0.28
trying to open library... ok
Finding cusparse
located at ~/.anaconda3/lib/libcusparse.so.7.0.28
trying to open library... ok
Finding cufft
located at ~/.anaconda3/lib/libcufft.so.7.0.35
trying to open library... ok
Finding curand
located at ~/.anaconda3/lib/libcurand.so.7.0.28
trying to open library... ok
Finding nvvm
located at ~/.anaconda3/lib/libnvvm.so.3.0.0
trying to open library... ok
finding libdevice for compute_20... ok
finding libdevice for compute_30... ok
finding libdevice for compute_35... ok
-------------------------------hardware detection-------------------------------
Found 1 CUDA devices
id 0 b'GeForce GTX 970' [SUPPORTED]
compute capability: 5.2
pci device id: 0
pci bus id: 7
Summary:
1/1 devices are supported
PASSED
Out[6]: True