Azure client manifest entry: n and r elements - manifest

While reviewing a client manifest provided by Azure Media Services for an HTTP Smooth Stream, I notice a new element (n) not found in previous IIS manifests and absent from Sam Zhang's blog.
According to previous manifests (clientManifestVersion 2.2), r means "repeat" and is used for compression - indicating duplicate fragment duration.
But by comparing two Azure manifests from the same stream at different times, you can see:
`<c t="868948936" d="2000" r="1770" n="136" />` // (# 8:21 PM)
`<c t="881664896" d="2000" r="1770" n="6494"/>` // (# 11:53 PM)
From what I understand,
d = 2000 indicates the fragment duration (2 seconds)
And where:
n1 = 136
n2 = 6494,
t1 = 868948936
t2 = 881664896,
n2 - n1 = 6358 * d = 12716000 + t1 = t2
Even though r is supposed to be a repeat, r remains the same while n increases over time... So what is r if it is unchanging, and what is n?

The n attribute is the zero-based index of the fragment, incremented by 1 for each new fragment. Just a meaningless counter: 0, 1, 2, 3, 4, ...
The r attribute indicates that r more fragments with the same duration follow the current fragment. It allows you to replace this:
<c t="1000" d="1000" />
<c t="2000" d="1000" />
<c t="3000" d="1000" />
<c t="4000" d="1000" />
With this much more compact representation:
<c t="1000" d="1000" r="3" />
You can think of it as just duplicating the XML element r number of times.
Edit: Based on the comment, I now understand the source of the confusion - the question is not actually about what these attributes are but why, with a live stream, does only n change as time goes along.
To understand this, you must understand how a live video is represented conceptually and how this differs from an on-demand video. The latter has a definite beginning and end, with a fixed number of fragments in between:
(start)123456789(end)
Whereas a live video by definition is one with no end - there may be a "last fragment" but new fragments are continually added to the end and what is currently the "last fragment" will change as time goes along:
(start)1234
(start)12345
(start)123456
Now this works all fine and super but you probably notice a problem here. Adaptive streaming technologies allow you to play any fragment of a video. If your video goes on, essentially, forever then the origin server must store an effectively infinite number of fragments! This cannot be allowed.
To solve this problem, adaptive streaming technologies introduce the concept of a DVR window - a sliding window over the video that contains all the data that can be viewed by players. Any data that slides out of range of this window can be discarded.
(start)[1]
(start)[12]
(start)[123]
(start)1[234]
(start)12[345]
(start)123[456]
(start)1234[567]
(start)12345[678]
(start)123456[789]
Let's discard the fragments we do not need and see how that looks. If your sliding window has a size 3 then the fragments visible to players would progress in time like this:
1
12
123
234
345
456
You notice that the size of the sliding window remains constant (once enough fragments are available to fill it) and that the index of the first fragment plus the sliding window size is sufficient to represent the entire sliding window.
There you have it: r is the number of fragments in the sliding window and n is the index of the first fragment! This is not the only way to represent live video but it is certainly the most efficient, due to the obvious small size of the data in the manifest.

Related

Cut ultrasound signal between specific values using Octave

I have an ultrasound wave (graph axes: Volt vs microsecond) and need to cut the signal/wave between two specific value to further analyze this clipping. My idea is to cut the signal between 0.2 V (y-axis). The wave is sine shaped as shown in the figure with the desired cutoff points in red
In my current code, I'm cutting the signal between 1900 to 4000 ms (x-axis) (Aa = A(1900:4000);) and then I want to make the aforementioned clipping and proceed with the code.
Does anyone know how I could do this y-axis clipping?
Thanks!! :)
clear
clf
pkg load signal
for k=1:2
w=1
filename=strcat("PCB 2.1 (",sprintf("%01d",k),").mat")
load(filename)
Lthisrun=length(A);
Pico(k,1:Lthisrun)=A;
Aa = A(1900:4000);
Ah= abs(hilbert(Aa));
step=100;
hold on
i=1;
Ac=0;
for index=1:step:3601
Ac(i+1)=Ac(i)+Ah(i);
i=i+1
r(k)=trapz(Ac)
end
end
ok, you want to just look at values 'above the noise' in your data. Or, in this case, 'clip out' everything below 0.2V. the easiest way to do this is with logical indexing. You can take an array and create a sub array eliminating everything that doesn't meet a certain logical condition. See this example:
f = #(x) sin(x)./x;
x = [-100:.1:100];
y = f(x);
plot(x,y);
figure;
x_trim = x(y>0.2);
y_trim = y(y>0.2);
plot(x_trim, y_trim);
From your question it looks like you want to do the clipping after applying the horizontal windowing from 1900-4000. (you say that that is in milliseconds, but your image shows the pulse being much sooner than 1900 ms). In any case, something like
Ab = Aa(Aa > 0.2);
will create another array Ab that will only contain the portions of Aa with values above 0.2. You may need to do something similar (see the example) for the horizontal axis if your x-data is not just the element index.

How can I use "Interpolated Absolute Discounting" for a bigram model in language modeling?

I want to compare two smoothing methods for a bigram model:
Add-one smoothing
Interpolated Absolute Discounting
For the first method, I found some codes.
def calculate_bigram_probabilty(self, previous_word, word):
bigram_word_probability_numerator = self.bigram_frequencies.get((previous_word, word), 0)
bigram_word_probability_denominator = self.unigram_frequencies.get(previous_word, 0)
if self.smoothing:
bigram_word_probability_numerator += 1
bigram_word_probability_denominator += self.unique__bigram_words
return 0.0 if bigram_word_probability_numerator == 0 or bigram_word_probability_denominator == 0 else float(
bigram_word_probability_numerator) / float(bigram_word_probability_denominator)
However, I found nothing for the second method except for some references for 'KneserNeyProbDist'. However, this is for trigrams!
How can I change my code above to calculate it? The parameters of this method must be estimated from a development-set.
In this answer I just clear up a few things that I just found about your problem, but I can't provide a coded solution.
with KneserNeyProbDist you seem to refer to a python implementation of that problem: https://kite.com/python/docs/nltk.probability.KneserNeyProbDist
There exists an article about Kneser–Ney smoothing on wikipedia: https://en.wikipedia.org/wiki/Kneser%E2%80%93Ney_smoothing
The article above links this tutorial: https://nlp.stanford.edu/~wcmac/papers/20050421-smoothing-tutorial.pdf but this has a small fault on the most important page 29, the clear text is this:
Modified Kneser-Ney
Chen and Goodman introduced modified Kneser-Ney:
Interpolation is used instead of backoff. Uses a separate discount for one- and two-counts instead of a single discount for all counts. Estimates discounts on held-out data instead of using a formula
based on training counts.
Experiments show all three modifications improve performance.
Modified Kneser-Ney consistently had best performance.
Regrettable the modified Version is not explained in that document.
The original documentation by Chen & Goodman luckily is available, the Modified Kneser–Ney smoothing is explained on page 370 of this document: http://u.cs.biu.ac.il/~yogo/courses/mt2013/papers/chen-goodman-99.pdf.
I copy the most important text and formula here as screenshot:
So the Modified Kneser–Ney smoothing now is known and seems being the best solution, just translating the description beside formula in running code is still one step to do.
It might be helpful that below the shown text (above in screenshot) in the original linked document is still some explanation that might help to understand the raw description.

How edge indexing works in graphhopper?

I'm here with a new question.
I'm making a custom algorithm that need precomputed data for the graph edges. I use the AllEdgesIterator like this :
AllEdgesIterator it = graph.getAllEdges();
int nbEdges = it.getCount();
int count = 0;
int[] myData = new int[nbEdges];
while (it.next())
{
count++;
...
}
The first weird thing is that nbEdges is equal to 15565 edges but count is only equal to 14417. How is it possible ?
The second weird thing is when I run my custom A* : I simply browse nodes using the outEdgeExplorer but I get an IndexOutOfBound at index 15569 on myData array. I thought that the edge indexes were included in [0 ; N-1] where N is the number of edges, is it really the case ?
What could be happening here ? By the way, I have disabled graph contraction hierarchies.
Thank you for answering so fast every time !
The first weird thing is that nbEdges is equal to 15565 edges
but count is only equal to 14417. How is it possible ?
This is because of the 'compaction' where unreachable subnetworks are removed, but currently only nodes are removed from the graph the edges are just disconnected and stay in the edges-'array' marked as deleted. So iter.getCount is just an upper limit but the AllEdgeIterator excludes such unused edges correctly when iterating and has the correct count. But using iter.getCount to allocate your custom data array is the correct thing to do.
Regarding the second question: that is probably because the QueryGraph introduces new virtual edges with a bigger edgeId as iter.getCount. Depending on the exact scenario there are different solutions like just excluding or using the original edge instead etc

Simulink Matlab function block deleting rows from a vector

That i want to do is to delete certain rows (or columns doesn't really mater...) from a given vector.
By going through Simulink's components found out that there is nothing performing such an operation,there are blocks help one add elements but nothing clearly for removing,so ended up trying to delete them by using a function block and following the online examples that demonstrate the usage of "[]".Lets say that i want to delete the second column of the vector u,i do u(:, 2) = [];.
That works absolutely fine in a separate m file or function but unfortunately not in a function block returning:
"Simulink does not have enough information to determine output sizes for
this block. If you think the errors below are inaccurate, try specifying
types for the block inputs and/or sizes for the block outputs."
and:
Size mismatch (size [4 x 4] ~= size [4 x 3]).
The size to the left is the size of the left-hand side of the assignment.
Function 'MATLAB Function' (#107.41.42), line 4, column 1:
"u"
Launch diagnostic report.
Is there any alternative you can suggest to remove several elements in a given vector in Simulink?
Thanks in advance
George
Finally,managed to do it without function block.There is a much easier way,by using Pad,and defining the output vector to be shorter than the input resulting in truncation.

more minimaler cubism.js horizon chart from json example

Following up on a previous question... I've got my minimal horizon chart example much more minimaler than before ( minimal cubism.js horizon chart example (TypeError: callback is not a function) )
<body>
<div class="mag"></div>
<script type="text/javascript">
var myContext = cubism.context();
var myMetr = myContext.metric(function(start, stop, step, callback) {
d3.json("../json/600s.json.php?t0=" + start/1000 + "&t1=" + stop/1000 + "&ss=" + step/1000, function(er, dt) {
if (!dt) return callback(new Error("unable to load data, or has NaNs"));
callback(null, dt.val);
});
});
var myHoriz = myContext.horizon()
.metric(myMetr);
d3.select(".mag")
.call(myHoriz);
</script>
</body>
The d3.json() bit calls a server side .php that I've written that returns a .json version of my measurements. The .php takes the start, stop, step (which cubism's context.metric() uses) as the t0, t1, and ss items in its http query string and sends back a .json file. The divides by 1000 are because I made my .php expect parameters in s, not ms. And the dt.val is because the actual array of my measurements is in the "val" member of the json output, e.g.
{
"other":"unused members...",
"n":5,
"val":[
22292.078125,
22292.03515625,
22292.005859375,
22292.02734375,
22292.021484375
]
}
The problem is, now that I've got it pared down to (I think) the bare minimum, AND I actually understand all of it instead of just pasting from other examples and hoping for the best (in which scenario, most things I try to change just break things instead of improving them), I need to start adding parameters and functions back to make it visually more useful.
Two problems first of all are, this measurement hovers all day around 22,300, and only varies +/- 10 maybe all day, so the graph is just a solid green rectangle, AND the label just says constantly "22k".
I've fixed the label with .format(d3.format(".3f")) (versus the default .2s which uses SI metric prefixes, thus the "22k" above).
What I can't figure out is how to use either axis, scale, extent, or what, so that this only shows a range of numbers that are relevant to the viewer. I don't actually care about the positive-green and negative-blue and darkening colours aspects of the horizon chart. I just used it as proof-of-concept to get the constantly-shifting window of measurements from my .json data source, but the part I really need to keep is the serverDelay, step, size, and such features of cubism.js that intelligently grab the initial window of data, and incrementally grab more via the .json requests.
So how do I keep the cubism bits I need, but usefully change my all-22300s graph to show the important +/- 10 units?
update re Scott Cameron's suggestion of horizon.extent([22315, 22320])... yes I had tried that and it had zero effect. Other things I've changed so far from "minimal" above...
var myHoriz = myContext.horizon()
.metric(myMetr)
.format(d3.format(".2f"))
.height(100)
.title("base1 (m): ")
.colors(["#08519c", "#006d2c"])
// .extent([22315, 22320]) // no effect with or without this line
;
I was able to improve the graph by using metric.subtract inserting it above the myHoriz line like so: (but it made the numerical label useless now):
var myMetr2 = myMetr.subtract(22315);
var myHoriz = myContext.horizon()
.metric(myMetr2)
.format...(continue as above)
All the examples seem so concise and expressive and work fine verbatim but so many of the tweaks I try to make to them seem to backfire, I'm not sure why that is. And similarly when I refer to the API wiki... maybe 4 out of 5 things I use from the API work immediately, but then I always seem to hit one that seems to have no effect, or breaks the chart completely. I'm not sure I've wrapped my head around how so many of the parameters being passed around are actually functions, for one thing.
Next hurdles after this scale/extent question, will be getting the horizontal time axis back (after having chopped it out to make things more minimal and easier to understand), and switching this from an area-looking graph to more of a line graph.
Anyway, all direction and suggestion appreciated.
Here's the one with the better vertical scale, but now the numerical label isn't what I want:
Have you tried horizon.extent? It lets you specify the [min, max] value for the horizon chart. By default, a linear scale will be created to map values within the extent to the pixels within the chart's height (specified with `horizon.height or default to 30 pixels).