I have a stream for a flash player that comes from H.264 Chunks - SPS, PPS, IDR and non-IDR. So far the FLV stucture is generated from the bytearray. The only problem left for me is to understand how I can get those Timestamp UI24 that FLVTag need.
To get the time for each FLVTag you do:
First get the frame rate from SPS file, you need to check for timing_info_present_flag to be 1;
timing_info_present_flag : 1
num_units_in_tick : 1000
time_scale :120000
fixed_frame_rate_flag : 1
So you count it like:
time_scale / (2 * num_units_in_tick) = frame_rate fps;
120000 /
2 * 1000 = 60 fps
After that you get the miliseconds per frame -> 1 / 60 = 0.01666(6)
So you count the frames (IDR and NDR) and you increase the timestamp accordingly. For SPS and PPS frame I drop the current timestamp to where we are currently.
Related
I am muxing H264 stream from v4l device to avi container using the approach outlined in the following stackoverflow question
The resulting files are playable, but for (lets say) 30 second recording the resulting file is only 10 seconds length. In other words, once I press 'Start recording' button until I press 'Stop' recording it is 30 seconds that have elapsed but file is only 10 seconds length (as shown in Windows Media player). Muxing starts immediately once I pressed the 'Start recording' button.
Any ideas on how could I approach this problem?
The problem was with the fps parameter:
AVStream *pst = avformat_new_stream(fc, 0);
vi = pst->index;
AVCodecContext *pcc = pst->codec;
_M;
avcodec_get_context_defaults3(pcc, AVMEDIA_TYPE_VIDEO);
pcc->codec_type = AVMEDIA_TYPE_VIDEO;
pcc->codec_id = codec_id;
pcc->bit_rate = br;
pcc->width = w;
pcc->height = h;
pcc->time_base.num = 1;
int fps = 30; // problem here
pcc->time_base.den = fps;
As it turned out, H264 stream produces frames with 13 fps. Once I made fps = 13, file duration become aligned with the expected time.
I want to retrieve the average fps of the measured performance recording.
So far I'm only able to get the duration and fps per frame by either hovering over the frame like this:
or by selecting the frame:
To get the average fps of all frames, I would have to sum and count them one by one by hand, which is quite inconvenient.
Firefox devtools for example displays the average fps at the top right of the panel.
You can use devtools-for-devtools.
Switch devtools to detached window mode (click devtools settings icon, click "undock" icon). Next time you can simply press Ctrl-Shift-D to toggle the mode.
Invoke devtools-for-devtools by pressing Ctrl-Shift-i
display FPS of all frames:
UI.panels.timeline._flameChart._model._frameModel._frames.slice(1).map(f => +(1000 / f.duration).toFixed(1))
display the average FPS:
+UI.panels.timeline._flameChart._model._frameModel._frames.slice(1).map(f => 1000 / f.duration).reduce((avg, fps, i) => (avg*i + fps) / (i+1), 0).toFixed(1)
You can save this code as snippets in devtools Snippets panel and invoke it after step 2 above.
I'd like to thank #wOxxOm for pointing out how to access DevTools for DevTools in the answer above.
However, the code given to calculate average FPS was not quite right. For example, if there's a frame that takes one second to render, then that frame's fps is one. If there is another frame that takes (1000 / 60) ms to render, then that frame's fps is 60. The original code would give an average fps of (60 + 1) / 2 for these two frames, which is incorrect.
The correct average fps is the total number of frames divided by the total duration. In this example, it is 2 / (1 + 1 / 60) fps.
One way to implement this is:
function averageFps() {
let frames = UI.panels.timeline._flameChart._model._frameModel._frames;
let duration = (frames[frames.length - 1].endTime - frames[0].startTime) / 1000;
let average = frames.length / duration
return +average.toFixed(2);
}
Just a quick note that the API seems to have changed slightly for this, so the new code to accomplish this is now:
let frames = UI.panels.timeline.flameChart.model.frameModelInternal.frames;
let frameSet = [];
let startTimeMs = UI.panels.timeline.flameChart.model.window().left;
let endTimeMs = UI.panels.timeline.flameChart.model.window().right;
let minFPS = 1000;
let maxFPS = -1;
let totalFPS = 0;
for (let frameIdx in frames) {
let frame = frames[frameIdx];
if (frame.startTime >= startTimeMs && endTimeMs >= frame.endTime) {
frameSet.push(frame);
let frameRate = (16.0/frame.duration) * 60;
if (maxFPS < frameRate) {
maxFPS = frameRate;
}
if (minFPS > frameRate) {
minFPS = frameRate;
}
totalFPS += frameRate;
}
}
console.log(`Total Frames: ${frameSet.length}`);
console.log(`Min FPS: ${minFPS}`);
console.log(`Max FPS: ${maxFPS}`);
console.log(`Average FPS: ${totalFPS / frameSet.length}`);
Updated #Daniel Le's solution that considers currently selected range
var startTime = UI.panels.timeline._flameChart._model._window.left;
var endTime = UI.panels.timeline._flameChart._model._window.right;
var frames = UI.panels.timeline._flameChart._model._frameModel._frames
.filter(frame => (frame.startTime > startTime) && (frame.endTime < endTime));
var duration = (frames[frames.length - 1].endTime - frames[0].startTime) / 1000;
var average = frames.length / duration
console.log(+average.toFixed(2));
Updated code:
display FPS of all frames:
UI.panels.timeline.flameChart.model
.frameModel().frames.slice(1).map(f => +(1000 / f.duration).toFixed(1))
display the average FPS:
UI.panels.timeline.flameChart.model.frameModel()
.frames.slice(1).map(f => 1000 / f.duration)
.reduce((avg, fps, i) => (avg*i + fps) / (i+1), 0).toFixed(1)
The devtools for devtools is great!
But your average does not take into account the duration of each frame nor the dropped or idle frames, given these durations [16,16,160] your code will report an average of 43.75 FPS, if you measure the weighted average correctly the FPS is 15.62.
// const frames = UI.panels.timeline.flameChart.model.frames().filter(({ idle }) => !idle);
const frames = [{ duration: 16 }, { duration: 16 }, { duration:160 }]
const totalDuration = frames.reduce((total, { duration }) => (total + duration), 0);
const weightedFps = frames.map(f => [f.dropped ? 0 : 1000 / f.duration, f.duration])
.reduce((avg, [fps, duration]) => avg + (fps * (duration / totalDuration)), 0);
console.log(weightedFps);
I'm trying to create a game with an independent frame rate, in which myObject is moving to the right at one unit per millisecond. However, I don't know how to calculate deltaTime in this code:
var currentTime = 0;
var lastTime = 0;
var deltaTime = 0;
while( play ) {
// Retrieve the current time
currentTime = Time.now();
deltaTime = currentTime - lastTime;
lastTime = currentTime;
// Move myObject at the rate of one unit per millisecond
myObject.x += 1 * deltaTime;
}
Let's say the first frame took 30 ms, so deltaTime should be 30 but it was 0
because we only know the time at the start of the frame not at the end of the frame. Then, in the second frame it took 40 ms, so deltaTime is 30 and thus myObject.x is 30. However, the elapsed time is 70 ms (30ms in 1st frame + 40ms in 2nd frame ) so myObject.x is supposed to be 70, not 30.
I'm not simulating physics, I'm just trying to move myObject relative to the elapsed time (not the frame).
How do I calculate deltaTime correctly?
I know that some game engine people use chunk of time or tick, so they're animating ahead of time. Also, I've already read Glenn Fiedler's article on fixing your timestep and many other ones, but I'm still confused.
Try this:
float LOW_LIMIT = 0.0167f; // Keep At/Below 60fps
float HIGH_LIMIT = 0.1f; // Keep At/Above 10fps
float lastTime = Time.now();
while( play ) {
float currentTime = Time.now();
float deltaTime = ( currentTime - lastTime ) / 1000.0f;
if ( deltaTime < LOW_LIMIT )
deltaTime = LOW_LIMIT;
else if ( deltaTime > HIGH_LIMIT )
deltaTime = HIGH_LIMIT;
lastTime = currentTime;
myObject.x += 1000 * deltaTime; // equivalent to one unit per ms (ie. 1000 per s)
}
There is alot wrong with this, but it makes it easier to illustrate the basic concept.
First, notice that you need to initialize lastTime with some value BEFORE the loop starts. You can use a lower value (i.e. Time.now() - 33) so that the first frame yields the desired delta, or just use it as I did (you will see that we limit it in the loop).
Next you get the current time at the start of each froame, use it to calculate the time elapsed since the last loop (which will be zero on the first run of this exaple). Then I like to convert it to seconds because it makes much more sense to work in "per second" than "per milisecond" - but feel free to remove the / 1000.0f part to keep it in ms.
Then you need to limit the deltaTime to some usable range (for the example I used 10-60fps, but you can change this as needed). This simply prevents the loop from running too fast or too slow. The HIGH_LIMIT is especially important because it will prevent very large delta values which can cause chaos in a game loop (better to be somewhat inaccurate than have the code break down) - LOW_LIMIT prevents zero (or very small) time steps, which can be equally problematic (especially for physics).
Finally, once you've calculated the new deltaTime for this frame, you save the current time for use during the next frame.
I have the following function that processes a buffer object containing a video frame supplied by GStreamer
def __handle_videoframe(self, appsink):
"""
Callback method for handling a video frame
Arguments:
appsink -- the sink to which gst supplies the frame (not used)
"""
buffer = self._videosink.emit('pull-buffer')
(w,h) = buffer.get_caps[0]["width"],buffer.get_caps[0]["height"]
reqBufferLength = w * h * 3 #Required buffer lenght for a raw rgb image of these dimensions
print "Buffer length: " + str(len(buffer.data))
print "Needed length: " + str(reqBufferLength)
img = pygame.image.frombuffer(buffer.data, self.vidsize, "RGB")
self.screen.blit(img, self.vidPos)
pygame.display.flip()
When running this code however, pygame crashes because the supplied buffer is larger than required and this size needs to match. I know this is probably caused by a faulty encoding of the movie that is played (as most movies do run fine), but is there a way to account for this contingency? Is there a way to resize the buffer on the go to a correct size? I have tried to just cut-off the tail of the buffer at the required length and then the movie does play, but the output is corrupted.
ok, a better solution was to use bufferproxies. They are less fuzzy about the length of the buffer.
img_sfc = pygame.Surface(video_dimensions, pygame.SWSURFACE, 24, (255, 65280, 16711680, 0))
img_buffer = img_sfc.get_buffer()
Then for each new frame:
img_buffer.write(buffer.data, 0)
pygame.display.get_surface().blit(img_sfc.copy(), vid_pos)
And voila, even incorrectly formatted buffers appear on screen without problems
One of my former students sent me a message about this interview question he got while applying for a job as a Junior Developer.
There are two candidates running for president in a mock classroom election. Given the two percentages of voters, find out the least amount of possible voters in the classroom.
Examples:
Input: 50.00,50.00
Output: 2
Input: 25.00,75.00
Output: 4
Input: 53.23, 46.77
Output: 124 // The first value, 1138 was wrong. Thanks to Loïc for the correct value
Note: The sum of the input percentages are always 100.00%, two decimal places
The last example got me scratching my head. It was the first time I heard about this problem, and I'm kindof stumped on how to solve this.
EDIT: I called my student about the problem, and told me that he was not sure about the last value. He said, and I quote, "It was an absurdly large number output" :( sorry! I should've researched more before posting it online~ I'm guessing 9,797 is the output on the last example though..
You can compute these values by using the best rational approximations of the voter percentages. Wikipedia describes how to obtain these values from the continued fraction (which can be computed these using the euclidean algorithm). The desired result is the first approximation which is within 0.005% of the expected value.
Here's an example with 53.23%:
10000 = 1 * 5323 + 4677
5323 = 1 * 4677 + 646
4677 = 7 * 646 + 155
646 = 4 * 155 + 26
155 = 5 * 26 + 25
26 = 1 * 25 + 1
25 = 25* 1 + 0
Approximations:
1: 1 / 1
-> 1 = 100%
2: 1 / (1 + 1/1)
-> 1/2 = 50%
2.5: 1 / (1 + 1 / (1 + 1/6))
-> 7/1 = 53.75%
3: 1 / (1 + 1 / (1 + 1/7))
-> 8/15 = 53.33%
3.5: 1 / (1 + 1 / (1 + 1 / (7 + 1/3)))
-> 25/47 = 53.19%
4: 1 / (1 + 1 / (1 + 1 / (7 + 1/4)))
-> 33/62 = 53.23%
The reason we have extra values before the 3rd and 4th convergents is that their last terms (7 and 4 respectively) are greater than 1, so we must test the approximation with the last term decremented.
The desired result is the denominator of the first value which rounds to the desired value, which in this vase is 62.
Sample Ruby implementation available here (using the formulae from the Wikipedia page here, so it looks slightly different to the above example).
First you can notice that a trivial solution is to have 10.000 voters. Now let's try to find something lower than that.
For each value of N starting à 1
For Each value of i starting à 1
If i/N = 46.77
return N
Always choose the minimum of the two percentages to be faster.
Or faster :
For each value of N starting à 1
i = floor(N*46.77/100)
For j = i or i+1
If round(j/N) = 46.77 and round((N-j)/N) = 53.23
return N
For the third example :
605/1138 = .5316344464
(1138-605)/1138 = .4683655536
but
606/1138 = .5325131810
(1138-606)/1138 = .4674868190
It can't be 1138...
But 62 is working :
33/62 = .5322580645
(62-33)/62 = .4677419355
Rounded it's giving you the good values.
(After some extensive edits:)
If you only have 2 voters, then you can only generate the following percentages for candidates A and B:
0+100, 100+0, or 50+50
If you have 3 voters, then you have
0+100, 100+0, 33.33+66.67, 66.67+33.33 [notice the rounding]
So this is a fun problem about fractions.
If you can make 25% then you have to have at least 4 people (so you can do 1/4, since 1/2 and 1/3 won't cut it). You can do it with more (i.e. 2/8 = 25%) but the problem asks for the least.
However, more interesting fractions require numbers greater than 1 in the numerator:
2/5 = 40%
Since you can't get that with anything but a 2 or more in the numerator (1/x will never cut it).
You can compare at each step and increase either the numerator or denominator, which is much more efficient than iterating over the whole sample space for j and then incrementing i;
i.e. if you have a percentage of 3%, checking solutions all the way up in the fashion of 96/99, 97/99, 98/99 before even getting to x/100 is a waste of time. Instead, you can increment the numerator or denominator based on how well your current guess is doing (greater than or less than) like so
int max = 5000; //we only need to go half-way at most.
public int minVoters (double onePercentage) {
double checkPercentage = onePercentage;
if (onePercentage > 50.0)
checkPercentage = 100-onePercentage; //get the smaller percentage value
double i=1;
double j=1; //arguments of Math.round must be double or float
double temp = 0;
while (j<max || i<max-1) { //we can go all the way to 4999/5000 for the lesser value
temp = (i/j)*100;
temp = Math.round(temp);
temp = temp/100;
if (temp == checkPercentage)
return j;
else if (temp > checkPercentage) //we passed up our value and need to increase the denominator
j++;
else if (temp < checkPercentage) //we are too low and increase the numerator
i++;
}
return 0; //no such solution
}
Step-wise example for finding the denominator that can yield 55%
55/100 = 11/20
100-55 = 45 = 9/20 (checkPercentage will be 45.0)
1/1 100.0%
1/2 50.00%
1/3 33.33%
2/3 66.67%
2/4 50.00%
2/5 40.00%
3/5 60.00%
3/6 50.00%
3/7 42.86% (too low, increase numerator)
4/7 57.14% (too high, increase denominator)
4/8 50.00%
4/9 44.44%
5/9 55.56%
5/10 50.00%
5/11 45.45%
6/11 54.54%
6/12 50.00%
6/13 46.15%
6/14 42.86%
7/14 50.00%
7/15 46.67%
7/16 43.75%
8/16 50.00%
8/17 47.06%
8/19 42.11%
9/19 47.37%
9/20 45.00% <-bingo
The nice thing about this method is that it will only take (i+j) steps where i is the numerator and j is the denominator.
I cannot see the relevance of this question to a position as junior developer.
Then answer that jumped into my head was more of a brute-force approach. There can be at most 5001 unique answers because there 5001 unique numbers between 00.00 and 50.00 . Consequently, why not create and save a look-up table. Obviously, there won't be 5001 unique answer because some answers will be repeated. The point is, there are only 5001 valid fractions because we are rounding to two digits.
int[] minPossible = new int[5001];
int numSolutionsFound = 0;
N = 2;
while(numSolutionsFound < 5001) {
for(int i = 0 ; i <= N/2 ; i++) {
//compute i/N
//see if the corresponding table entry is set
//if not write N there and increment numSolutionsFound
}
N++;
}
//Save answer here
Now the solution is merely a table look up.
FWIW I realize the euclidean solution is "correct". But I'd NEVER come up with that mid interview. However, I'd know something like that was possible -- but I won't be able to whip it out on the spot.