I am trying to run a script example from the huggingface documentation:
import torch
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained('gpt2')
generated = tokenizer.encode("The Manhattan bridge")
context = torch.tensor([generated])
past = None
for i in range(100):
print(i)
output, past = model(context, past=past)
token = torch.argmax(output[..., -1, :])
generated += [token.tolist()]
context = token.unsqueeze(0)
sequence = tokenizer.decode(generated)
print(sequence)
But I have an error:
TypeError: forward() got an unexpected keyword argument 'past'
What should I change to use 'past'?
Try updating past to past_key_values. I believe the documentation has been changed.
Related
I want to use the embedding layer as the input for encoder, however I got an error as follow. My input y is a time series data with shape of 1*84. Could you please help me with that?
import numpy
import torch.nn as nn
r_input = torch.nn.Embedding(84, 10)
activation = nn.functional.relu
mu_r = nn.Linear(10, 6)
log_var_r = nn.Linear(10, 6)
y = np.random.rand(1,84)
def encode_r(y):
y = torch.reshape(y, (-1, 1, 84)) # torch.Size([batch_size, 1, 84])
hidden = torch.flatten(activation(r_input(y)), start_dim = 1)
z_mu = mu_r(hidden)
z_log_var = log_var_r(hidden)
return z_mu, z_log_var```
Error: RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)
According to this thread: https://discuss.pytorch.org/t/expected-tensor-for-argument-1-indices-to-have-scalar-type-long-but-got-cpufloattensor-instead-while-checking-arguments-for-embedding/32441/4, it seems that one possible solution would be to ensure embeddings have integer values and not float values in them (by embeddings we mean the lookup table not an actual embedding vector).
I run the following to train a wrapped model from some task. I want to get the hyper-parameters from the wrapper. I tried the following
library(mlr)
lrn = makeLearner("classif.ksvm")
lrn = makeRemoveConstantFeaturesWrapper(lrn)
df = getTaskData(sonar.task)
df$constant = 1
task = makeClassifTask(data = df, target = "Class")
model = train(learner = lrn, task = sonar.task)
model
getHyperPars(model)
I got the following message.
Error in UseMethod("getHyperPars") :
no applicable method for 'getHyperPars' applied to an object of class "c('PreprocModel', 'BaseWrapperModel', 'WrappedModel')"
How can I get and set hyper-parameters for a wrapped model?
Thanks!
You have to apply it on the learner:
getHyperPars(lrn)
This only gives hyperparameters that are set explicitly.
Is it possible to read and access the solver properties in Pycaffe?
I need to use some of the information stored in the solver file, but apparently the solver object which is created using
import caffe
solver = caffe.get_solver(solver_path)
is of no use in this case. Is there any other way to get around this problem?
I couldn't find what I was after using solver object and I ended up writing a quick function to get around this issue:
def retrieve_field(solver_path, field=None):
'''
Returns a specific solver parameter value using the specified field
or the whole content of the solver file, when no field is provided.
returns:
a string, as a field value or the whole content as list
'''
lines = []
field_segments = []
with open(solver_path, 'r') as file:
for line in file:
line = line.strip()
lines.append(line)
field_segments = line.split(':')
if (field_segments[0] == field):
#if that line contains # marks (for comments)
if('#' in field_segments[-1]):
idx = field_segments[-1].index('#')
return field_segments[-1][0:idx]
else:
return field_segments[-1]
return lines
I am exploring support vector machine regression with this tutorial http://www.svm-tutorial.com/2014/10/support-vector-regression-r/. I encounter Error in do.call(method, c(list(train.x, data = data, subset = train.ind[[sample]]), : 'what' must be a function or character string when tuning the model using grid search but I am not sure what that means. Here are the codes I used:
svm_m <- svm(FuelRate~ Heading+Distance+Elevatio+YieldDry+HarvestM, data = fr_tr)
tuneResult <- tune(svm_m, FuelRate ~ Heading+Distance+Elevatio+YieldDry+HarvestM, data = fr_tr, ranges = list(epsilon = seq(0,1,0.1), cost=2^(2:9))). Can anyone teach me about the solution?
Sorry, I used a wrong argument here tuneResult <- tune(svm_m, FuelRate ~ Heading+Distance+Elevatio+YieldDry+HarvestM, data = fr_tr, ranges = list(epsilon = seq(0,1,0.1), cost=2^(2:9))). The first argument should be method = svm.
I try to use Pool from the multiprocessing module to speed up reading in large csv files. For this, I adapted an example (from py2k), but it seems like the csv.dictreader object has no length. Does it mean I can only iterate over it? Is there a way to chunk it still?
These questions seemed relevant, but did not really answer my question:
Number of lines in csv.DictReader,
How to chunk a list in Python 3?
My code tried to do this:
source = open('/scratch/data.txt','r')
def csv2nodes(r):
strptime = time.strptime
mktime = time.mktime
l = []
ppl = set()
for row in r:
cell = int(row['cell'])
id = int(row['seq_ei'])
st = mktime(strptime(row['dat_deb_occupation'],'%d/%m/%Y'))
ed = mktime(strptime(row['dat_fin_occupation'],'%d/%m/%Y'))
# collect list
l.append([(id,cell,{1:st,2: ed})])
# collect separate sets
ppl.add(id)
return (l,ppl)
def csv2graph(source):
r = csv.DictReader(source,delimiter=',')
MG=nx.MultiGraph()
l = []
ppl = set()
# Remember that I use integers for edge attributes, to save space! Dic above.
# start: 1
# end: 2
p = Pool(processes=4)
node_divisor = len(p._pool)*4
node_chunks = list(chunks(r,int(len(r)/int(node_divisor))))
num_chunks = len(node_chunks)
pedgelists = p.map(csv2nodes,
zip(node_chunks))
ll = []
for l in pedgelists:
ll.append(l[0])
ppl.update(l[1])
MG.add_edges_from(ll)
return (MG,ppl)
From the csv.DictReader documentation (and the csv.reader class it subclasses), the class returns an iterator. The code should have thrown a TypeError when you called len().
You can still chunk the data, but you'll have to read it entirely into memory. If you're concerned about memory you can switch from csv.DictReader to csv.reader and skip the overhead of the dictionaries csv.DictReader creates. To improve readability in csv2nodes(), you can assign constants to address each field's index:
CELL = 0
SEQ_EI = 1
DAT_DEB_OCCUPATION = 4
DAT_FIN_OCCUPATION = 5
I also recommend using a different variable than id, since that's a built-in function name.