IDA SDK Loader: memory mapping - reverse-engineering

I'm trying to write a loader for a file type I have. It's basically a binary file that has a bunch of bootloader ROM data in it(multiple bootloaders). I can parse the file and load all the BLs into the database just fine. My problem is that each bootloader sometimes loads data from inside its address range. So for example, if I load the 3rd bootloader into the database and then examine its code I might see it try to access some data at 0x10 into its own range. When I hover over that 0x10 and or double click it I am taken to 0x10 at the start of the database and not the start of that specific bootloader. Is there any way I can get it to show me what is at 0x10 in its own memory range?
I have each BL in its own code segment.
So I'd basically like each BL segment to act like its own database.
To help explain it better, lets say I have BL1 from 0 - 0x1000, BL2 from 0x1000 - 0x2000 and BL3 from 0x2000 - 0x3000 in the database.

Related

Reverse engineer of APC UPS serial protocol

I have a APC SMC1000-2UC UPS device that has a serial port to connection. The serial port protocol seems that is based on microlink protocol which has not documented. I monitored the communication of the UPC and PC witch UPS deriver has been initialed. I want to detect command of UPS such as shutdown command by a microcontroller-based device. Some information of "https://sites.google.com/site/klaasdc/apc-smartups-decode" site is compatible with things I observed. but calculation of frame checksum and Challenge string calculation don't pass.
Data length of protocol has been set to 32 bytes, So each frame has 35 bytes.
[Msg ID | 32 byte data | 2 byte checksum]
Regarding calculation of challenge frame, the UPS send 0x65 frame ID then 0x68 frame ID, after that the PC response with 0x65 frame ID and UPS send confirmed frame by 0x65 frame ID. based on presented calculation, I think format or Position of Password_1, Header data and two bytes of that has been changed as the protocol has been configured to 32 bytes data. The following frame are a sample of this challenge:
Header frame: 0x00 0a206903fa27090001004000f802fe04fe0940fc1042fc1044fc20f80416fc10 32a6
UPS : 0x65 ffff00010000a0e80000 c0bbb4e1 000001040000001000000004000000000020 7350
UPS : 0x68 000000000000000000000008004c2943000000000966039a063b675601f30000 864f
PC : 0x65 0a 04 8afb65f1 bdf0
UPS : 0x65 ffff000100000eaf62d8 8afb65f1 000001040000001000000004000000000020 6227
How can I satisfy the challenge and checksum type? I try many type of checksum for that data but they not correct.
It may be a bit late, but have you looked at this:
https://github.com/klaasdc/apcups-serial-test
This looks like somebody got pretty far reverse engineering the MicroLink protocol, including the checksum part. The GitHub repo also contains a link to a web page with a protocol description.

Big binary data share between processes

I have a big binary data iof ip data about Xmb. Processes use binary do some search algorithm to lookup ip address. I have three method.
1. put in ets. but I suppose every read access will copy big binary to process. :(
2. put in gen_server state. processes use gen_server:call to get address.The short coming concurrency.
3. compile binary into beam. but when I compile get
eheap_alloc: Cannot allocate 1318267840 bytes of memory (of type "heap")
which the best practice of big data share in erlang?
Binaries over 64 bytes in size are stored as reference counted binaries and their data is stored outside the heap of any process. If such a binary is sent to any process, the underlying data is not duplicated. So, if you store such a binary in an ETS table and then access it from various processes, the underlying data will not be copied, only its reference count will be incremented/decremented. I'd suggest going with the ETS table solution.
Here's a demonstration of the memory usage at boot, after inserting a 100MB binary into an ETS table, and after fetching a copy of the binary into the shell process. The memory usage does not change after we have a copy binary stored in the shell process. The same would not be true if it was million character string (list of integers) that we were copying in from ETS or another process.
1> erlang:memory().
[{total,21912472},
{processes,5515456},
{processes_used,5510816},
{system,16397016},
{atom,223561},
{atom_used,219143},
{binary,844872},
{code,4808780},
{ets,301232}]
2> ets:new(foo, [named_table, set]).
foo
3> ets:insert(foo, {foo, binary:copy(<<".">>, 104857600)}).
true
4> erlang:memory().
[{total,127038632},
{processes,5600320},
{processes_used,5599952},
{system,121438312},
{atom,223561},
{atom_used,220445},
{binary,105770576},
{code,4908097},
{ets,308416}]
5> X = ets:lookup(foo, foo).
[{foo,<<"........................................................................................................"...>>}]
6> erlang:memory().
[{total,127511632},
{processes,6082360},
{processes_used,6081992},
{system,121429272},
{atom,223561},
{atom_used,220445},
{binary,105761504},
{code,4908097},
{ets,308416}]
You can find a lot more info about how to efficiently work with binaries in Erlang in the link above.

8051 External /WR signal not generated when UART bootloader is used after reset

Goal of the project: To connect an LCD with 8051 as an external memory-mapped I/O device
Problem: Given the following details, my 8051 controller just does not generate an external RD/WR command as required for the rest of the code to work.
Previous work: I used 8051 port 3 pins to generate EN, R/W and RS signals and got it to work. Therefore, I know that my command sequence is working fine. However, this was a really inefficient way of using the LCD because the enable pulse was generated by setting and resetting a port pin. I wish to connect the LCD using the external WR/RD signals and mapping it as a memory-mapped IO device. I have worked through the timing diagrams and the overall block diagram is attached here. As you can see (in the block diagram), the R/W line of LCD is activated using the most significant 6 pins of port 2 so that the LCD gets activated only at the right memory addresses. This operation (implemented in an SPLD) also serves to ensure the delay required at the LCD to ensure the minimum setup time after Port 2 pins 0,1 are used to set inputs at R/W and RS signals of LCD.
Additional hardware info: I have attached a spice diagram to show how the rest of my 8051 is connected. The one thing that is not included there is this: "I use a momentary pushbutton and pull-down resistor for /PSEN, and hold that button when coming out of reset in order to force bootloader operation; then, after the bootloader has started, I release that button to eliminate drive fight issues on the /PSEN line. I use a header/jumper for the /EA input to ensure it is high. Note that if you use these hardware conditions to enter the bootloader when you come out of reset, then the Atmel bootloader is entered regardless of the values of BLJB, BSB, and SBV."
Software used: I am using the paulmon2 to test my code. Programming is done using Flip utility: Flip 3.4.7 through the serial port. A serial emulator program (TeraTerm) is used to communicate with the microcontroller. The microcontroller first executes the paulmon code as well as its extra commands that have been programmed into it before the current user code at 0x2000 location. An extra command allows the user to jump to this code using 'J' command and then giving the address for memory: 0x2000. This calls the current
program and executes it. This is where my code resides and executes from.
The addresses used to map LCD are the following:
LCD_INSTR_WR: 0xA8FF ---> Used to write commands to LCD controller.
This includes all initialization and setup and management commands.
LCD_INSTR_RD: 0xA9FF ---> Used to read command. Done only to read the busy
flag or the current address counter. This is valid only for a single
instruction on the LCD.
LCD_DATA_WR: 0xAAFF ---> Used to write Data to the current address which has
been set either in DDRAM or CGRAM given the LCD_INSTR_WR above.
LCD_DATA_RD: 0xABFF ----> Used to read Data from the current address which
has been set either in DDRAM or CGRAM given the LCD_INSTR_WR above.
The code snippet I write in C to write the external memory:
//Global variables
volatile unsigned char xdata *LCD_INSTR_WR = (char xdata *) 0xA8FF;
volatile unsigned char xdata *LCD_INSTR_RD = (char xdata *) 0xA9FF;
volatile unsigned char xdata *LCD_DATA_WR = (char xdata *) 0xAAFF;
volatile unsigned char xdata *LCD_DATA_RD = (char xdata *) 0xABFF;
/// More code
//Write command example
lcdbusywait();
* LCD_DATA_WR = cc;
Earlier tests one to figure out the problem:
I have tried writing to the memory locations above 2000 using the paulmon memory edit instructions and they write the memory locations alright. Even /WR command is generated in this case as observed (but I have not properly measured/counted the accesses and /WR edge changes.
I have used the logic analyser to confirm that the address (and consequently RS and RW) and data (0x30H command in the beginning) are coming to the ports as expected. ALE is being generated.
I have verified that AUXR register bit EXTRAM is set (AUXR = 0x0E). Also, since EXTRAM is set by default, I tried to remove my initialization code for AUXR completely and that didn’t work either.
I was not sure that the C code that I have written for the XRAM address accesses is correct. However, I went on to check the .asm file and (unless I am neglecting something very minute), the assembly code generated does assign a 0x30h value as immediate data to a register A and uses a “MOVX #dptr,A” instruction to write this value to external memory.
Finally, this is my first post at Stack overflow so the formatting may be off and I do realize this is an extremely long post. Apologies for that. Let me know if you need to see the code files or the compiled hex file or other details. All your help is deeply appreciated.
volatile unsigned char xdata *LCD_INSTR_WR = (char xdata *) 0xA8FF;
I guess LCD_INSTR_WR should have address0xA8FF value.
You can try by using
#define LCD_INSTR_WR XBYTE[0xA8FF]
and then
LCD_INSTR_WR = cc;
or
char xdata LCD_INSTR_WR _at_ 0xA8FF; //This will declare the LCD_INSTR_WR at location 0xA8FF-
You may need to look into the data sheet of the micro controller how to configure extrnal memory
LCD_INSTR_WR = cc;

Bulk export of binary waveform data from oscilloscope to data points (csv preferred)

I'm working with some binary waveform files from various early to mid-90's HP scopes. I am trying to do a bulk conversion (we have over 5000) of the files to CSV's and then upload them into a database. I've tried hexdump, xxd, od, strings, etc. and none of them seem to work. I did hunt down a programmers manual but it's not making a whole lot of sense.
The files have a preamble line as ascii text but then the data points are in binary and for some reason nothing I try can decode them. The preamble gives the data necessary to use the binary values and calculate the correct values. It also states that the data is in WORD format.
:WAV:PRE 2,1,32768,1,+4.000000E-08,-4.9722700001108E-06,0,+2.460630E-04,+2.500000E+00,16384;:WAV:DATA #800065536^W�^W�^W�^
I'm pretty confused.
Have a look at
http://www.naic.edu/~phil/hardware/oscilloscopes/9000A_Programmer_Reference.pdf
specifically page 1-21. After ":WAV:DATA", I think the rest of the chunk above will have 65536 8-bit data bytes (the start of which is represented above by �) . The ^W is probably a delimiter, so you would have to parse that out. Just a thought.
UPDATE: I'm new to oscilloscope data collection and am trying to figure the whole thing out from scratch. So, on further digging, it looks like the data you have provided shows this:
PREamble:
- WORD format (16-bit signed integers split into 2 8-bit bytes)
- If there is a WAV:BYT section, that would specify byte order for each pair
- RAW data
- 32768 data points
- COUNT = 1 (I'm not clear on the meaning of this)
- Next 3 should be X increment, origin, reference
- Next 3 should be Y increment, origin, reference, although the manual that I pointed you at above has many more fields than just these, so you might want to consult your specific scope manual.
DATA:
- On closer examination, I don't think the ^W is a delimiter, I think it is the first byte of the pair (0010111). The � character is apparently a standard "I don't know how to represent this character" web representation. You would need to look at that character as 8 bits also.
- 65536 byte pairs of data
I'm not finding a utility that will do this for you. I think you're going to have to write or acquire some code (Perl, C, Java, Python, VB, etc.) to get this done.

VHDL MIPS 5 stage pipeline Bug

The code for this is too long to post so Ill just describe it. I've created a 5 stage mips pipe that almost works. The catch is that EVERY lw instruction that reaches the instruction decode stage overwrites the control signal values in the execution stage. Not only that it causes the PC to skip can instruction, i.e from 300 -> 308. I just need some idea on where to look for bugs since this is a class assignment. If we take out all the LW instructions the CPU works fine.
Example:
The adder in the EX stage is going to sub $4 $1 $2 which should be 1
Once LW enters the ID stage ALUsrc is asserted AND ALUop is changed from subtract to add
This forces the adder in the EX stage to add $4 $1 $2 resulting in 5 being stored in $4
http://en.wikipedia.org/wiki/File:MIPS_Architecture_%28Pipelined%29.svg
The MIPS 5 Stage Pipeline (annotated to show Write Reg Select and enable)
The bottom line through the pipeline stages represents the register file write (back) port address and write enable and WB is the data from memory.
http://www.mrc.uidaho.edu/mrc/people/jff/digital/MIPSir.html
Load Word Instruction
Description:
A word is loaded into a register from the specified address.
Operation: $t = MEM[$s + offset]; advance_pc (4);
Syntax: lw $t, offset($s)
Encoding:
1000 11ss ssst tttt iiii iiii iiii iiii
Where the write register address ($t) input is read from data memory address comprised of register file register $s offset with the immediate value i which gets sign extended. Your $4 is $t above, $1 or $2 is $s while the remaining register file output lane sounds to be suborned for the sign extended immediate.
From your description it sounds like you aren't using a three port register file with one port a write only port.
With a three port register file the only time you run into conflicts is when you attempt to use the new register file value from memory before it is read from memory and written to the register file. That can be managed by a compiler scheduling NOOPs until the outstanding register file write is retired when a following instruction is trying to use it, or stalling the IF/ID in hardware when it's output contains a reference to an outstanding register file write.
There are three instructions that can be in flight to the right of IF/ID, each with a write to register file address and a write enable. You'd need to compare both instruction decode register file addresses to all three of those and stall IF/ID until those clear out. The write enable stored in each of those three pipeline stages are used to determine whether the write register address in those pipeilne stages should be compared.
Because the ID/EX, EX/MEM and MEM/WB write register file addresses are not used anywhere else the circuitry for doing the comparison can be collocated with IF/ID and the Register File, preventing unnecessary layout delays affecting the minimum clock cycle.
Using a two port register file is much simpler and infers IF/ID stalling until the write enable comes back from MEM/WB, effectively turning any memory reading instructions into 3 cycle instructions (or more, data memory can stall if it's a cache or slow). It makes a three port register file more or less necessary for performance reasons. There's an implied multiplexer to source for at least one of the two register file port controls (write enable, write address) from the MEM/WB stage when IF/ID is stalled (for memory->regfile).
Data memory access can stall MEM/WB, just like instruction memory access can also stall IF/ID. A stalled IF/ID doesn't issue a write enable for the register file to ID/EX nor does a stalled MEM/WB.