I am using vowpalwabbit in contextual bandit settings. But I am struck with a strange issue where vowpalwabbit just generating same PMF irrespective of context. Ideally it should generate different PMFs for action selection based on different context. Here is the sample data I am using.
shared |Context t1=a_c t2:5 t3=a_b t4:2 t5:10
|Action arm=a1
|Action arm=a2
|Action arm=a3
|Action arm=a4
0:-5:0.09 | Action arm=a5
|Action arm=a6
|Action arm=a7
|Action arm=a8
|Action arm=a9
|Action arm=a10
|Action arm=a11
I initialized my vowpalwabbit with following setting.
--cb_explore_adf --cb_type mtr --epsilon 0.05
Here is the action distribution irrespective of context in data.
Action Dist. in data
Action Dist. of Contextual Bandit
Wondering what could be the cause of vowpalwabbit saturating. Is it something with the hyperparams provided?
--cb_explore_adf --cb_type mtr -q CA --epsilon 0.05 worked for me.
Related
I am using the Transformer model from Hugging face for machine translation. However, my input data has relational information as shown below:
I want to craft a graph like the like the following:
________
| |
| \|/
He ended his meeting on Tuesday night.
/|\ | | /|\
| | | |
|__| |___________|
Essentially each token in the sentence is a node and there could be an edge embedded between the tokens.
In a normal transformer, the tokens are processed into token embeddings, also there is an encoding of each position which resulted into positional embeddings.
How could I do something similar with the edge information?
Theoretically I could take the edge type and the positional encoding of a node and output an embedding. The embeddings of all the edges can be added to the positional embeddings for the corresponding nodes.
Ideally, I would like to implement this with the hugging face transformer.
I am struggling to understand how could I update the positional embedding here:
https://github.com/huggingface/transformers/blob/main/src/transformers/models/longformer/modeling_longformer.py#L453
self.position_embeddings = nn.Embedding(
config.max_position_embeddings, config.hidden_size, padding_idx=self.padding_idx
)
Your question is generally about how to use linguistic representations to enrich word representations in Transformer language models. This is an open research question with no clear answer. One option given by Prange et al. (2022) is to define a function which returns a subgraph of the whole-sentence graph given a particular token, and to process the token's subgraph so that it is represented in a fixed-length vector which can then be readily combined with the token's representation for use in the rest of the Transformer LM. Their related work section reviews other approaches and is worth a look.
I am starting to work with Vowpal Wabbit with Python and I am kinda struggling with its lack of documentation.
Do you guys know what modeling it uses as a cost/reward estimation for each arm? Do you know how to retrieve this current estimation?
vw = pyvw.vw("--cb_explore 2 --epsilon 0.2")
input = "2:-20:0.5 | Anna"
vw.learn(initial_input)
input = "1:-10:0.1 | Anna"
vw.learn(initial_input)
vw.predict(" | Anna")
Output would be:
[0.10000000149011612, 0.9000000357627869]
How can I also get the expected value for each arm? Something like
[-10.00, -20.00]
When using _explore you get back a PMF over the given actions. This is true for CB and CB_adf.
However, when using the non-explore version for each of those things differ a bit.
--cb is going to give you the chosen action directly, whereas --cb_adf is going to return the score for each given action.
So in this situation changing to using action dependent features (ADF) should provide the score/estimated cost.
So I'm doing some hobby-related stuff which involves taking Fourier transforms of large real arrays which barely fit in memory, and was curious to see if there was an in-place version of rfft and irfft that saved RAM, since RAM consumption is important to me. These transforms are possible despite the input-vs-output-type mismatch, and require an extra row of padding.
In Implement in-place rfft! and irfft!, Tim Holy said he was working on an in-place rfft! and irfft! that made use of a buffer-containing RCpair object, but then Steven Johnson said that he was implementing something equivalent using A_mul_B!(y, plan, x), which he elaborated on here.
Things get a little weird from then on. In the documentation for both 0.3.0 and 0.4.0 there is no mention of A_mul_B!, although A_mul_B is listed. But when I try entering them into Julia, I get
A_mul_B!
A_mul_B! (generic function with 28 methods)
A_mul_B
ERROR: A_mul_B not defined
which suggests that the situation is actually the opposite of what the documentation currently describes.
So since A_mul_B! seems to exist, but isn't documented anywhere, I tried to guess how to test it in-place as follows:
A = rand(Float32, 10, 10);
p = plan_rfft(A);
A_mul_B!(A,p,A)
which resulted in
ERROR: `A_mul_B!` has no method matching A_mul_B!(::Array{Float32,2}, ::Function, ::Array{Float32,2})
So...
Are in-place real FFTs still a work in progress? Or am I using A_mul_B! wrong?
Is there a mismatch between the 0.3.0 documentation and 0.3.0's function library?
That pull request from Steven Johnson is listed as open, not merged; that means the work hasn't been finished yet. The one from me is closed, but if you want the code you can grab it by clicking on the commits.
The docs indeed omit mention of A_mul_B!. A_mul_B is equivalent to A*B, and so isn't exported independently now. A_mul_B! would be used like this: instead of C = A*B, you could say A_mul_B!(C, A, B).
Can you please edit the docs to fix these issues? (You can edit files here in your webbrowser.)
I have PAGENAME
For example PAGE: iPad
I need two parameters for this page:
Parameter PRICE: $999
Parameter AVAILABLE: Available, Not available
How can I get there parameters on page?
I would like to be able to get, in addition than {{PAGENAME}}, parameters on the page ?
And where could I store a lot of these parameters?
For example:
iPad1 — $999 — Available
iPad2 — $1999 — Available
iPad3 — $2999 — Not available
iPad4 — $3999 — Available
iPad5 — $2999 — Not available
etc.
How is it edit more comfortable for a lot of parameters?
I work on the Pixel Dungeon Wiki a lot and it has a Data template which can be used to store information about an item.
To use it, copy the template over then create a page called ipad/data.
Then add to ipad/data:
{{Data|{{{1|}}}|{{{2|}}}|{{{3|}}}|{{{4|}}}|{{{$|}}}
| iPad1price = $999
| iPad1available = AVAILABLE
| iPad2price = $1999
| iPad2available = AVAILABLE
...
}}
Then you can access the information on the page with:
iPad1 price is {{:ipad/data||iPad1price}}
The Data template documentation is quite good for more info on this
I'm trying to created a template that will output a dynamic report by generating a list of pages with Dynamic Page List and then run queries on each page using Semantic Mediawiki, outputting the whole mess in a table. The structure of the template looks like this:
{| class="wikitable sortable collapsible collapsed" width="100%"
|- valign="bottom"
! width="15%" | Compound ID
! width="15%" | Generic name
! width="15%" | Trade name
! width="15%" | Developer
! width="10%" | Phase of development
! width="30%" | Target
{{#dpl:
| category=compounds
| linksto={{PAGENAME}}{{#dpl: linksto={{PAGENAME}} | redirects=only | mode=userformat | format=,{{!}},%TITLE%, | suppresserrors=true }}
| ignorecase=true
| distinct=strict
| format=,{{!}}- valign="top"\n{{indication-based compound list/row|%TITLE%}}\n
| noresultsheader={{!}}-\n{{!}} colspan="6" align="center" {{!}} ''No compounds have yet been studied for this indication.''
}}
|}
{{indication-based compound list/row}} consists of the following:
<includeonly>{{!}} {{#show: {{{1}}} | ?compound id }}
{{!}}{{!}} {{#show: {{{1}}} | ?compound generic name }}
{{!}}{{!}} {{#show: {{{1}}} | ?compound trade name }}
{{!}}{{!}} {{#show: {{{1}}} | ?developer }}
{{!}}{{!}} {{#ask: [[category:clinical studies]][[compound number::{{{1|}}}]] | ?Study phase=| limit=1| mainlabel=-| searchlabel=| sort=Study phase| order=descending }}
{{!}}{{!}} {{#show: {{{1|}}} | ?target }}</includeonly>
The {{#ask}} query is the reason why this isn't just a straight-forward DPL table: because the current phase of clinical development of a specific compound for a specific indication is a dynamic piece of data that changes frequently, the only option seems to be using an {{#ask}} query to find out the highest value for study phase among the clinical trials that meet the parameters. But this template will be surveying auto-generated pages and I don't have a way of auto-loading a list of indications for each compound (to apply semantic markup to), so instead it's generating based on lists of indication links that are themselves generated by a {{#dpl}} function on each compound page.
I've walked through this step by step, and none of the following are the problem:
The template code is sound, and generates the proper structure.
The table row template is sound, and if I remove the {{#dpl}} function and replace it with the template and a compound page name as a parameter, e.g. {{indication-based compound list/row|infliximab}}, it creates exactly the output I'm looking for.
The {{#dpl}} function produces the correct list of pages when using the default output format.
The {{#dpl}} function passes the list as sequential parameters properly when using format = ,{{template}} with a simply dummy template (just an unordered list).
However, when I put all the pieces together I just get empty table cells. The {{#show}} and {{#ask}} functions can't seem to accept the DPL input as parameters. Is there an additional argument or intermediate step I'm skipping that can sanitize the input and make the whole thing work? Or are DPL and SMW fundamentally incompatible in this way?
DynamicPageList 2.01
Semantic MediaWiki 1.8
MediaWiki 1.20.2
PHP 5.3.19 (apache2handler)
MySQL 5.1.30
Looks like the problem is that Semantic Mediawiki can't parse the PAGE% and %TITLE% variables generated by Dynamic Page List when you use the format argument, and there's no way around that since DPL doesn't render those into real data until the function is completed.