[Rap files] Games installing on PS3 enquiry; THREAD: [Rap files]. Jon Snow Di DonDadda. 2,958 Posts 45,214 Reputation 5,968 Credits Former Staff.
1EDAT Stuff
EDAT Stuff[edit]
My WIP on decryting EDATs. I've found 4 likely keys used in EDAT processing inside appldr (included in unedat.c).Mostly non-working unedat
JuanNadie[edit]
Actually an EDAT has several keys. One is in the SELF, others are at the rif and others on firmware. That an EDAT works on lower firmware only proves that the firmware has those keys.
BTW I have tricked the PS3 into decrypt an EDAT (Buzz). The procedure involves knowing how the NPD element is used (most of this is already published).
EXE.trim.ALL[edit]
Actually as I know:
and uint64_t unknown4 is:
JuanNadie[edit]
I haven´t fully reversed the EDAT algorithm (I’m missing bytes from 0xB0 to 0xFF) but there is enough info for a post. The explanation is for those with flag (offset 0x80-0x83) equals to 0xC which is a normal EDAT. For others values minor changes should be made. Let’s begin.
USER SPACE:(Probably incomplete (enough for decryption), most of it is guessed)On previous post I mentioned the command useds by developers to open/read an EDAT. That command has 3 parts:
Verify: This parts verifies the NPD element by checking that user is authorized to open the file (using the devklic). In addition it calls LV2 to create an entry in the memory (probably using syscall471) so for further steps it will use the devklic (free) or the decrypted rif key (paid) (see SELF algorithm). I’ll call this rifkey on the rest of the document.
Open the file: This will call kernel space. Performs several checks and creates an entry on another table.
Read: It also calls LV2, which will decrypt and send data back.
<KERNEL SPACE:
When an open request is received:
Read the first 0x100 bytes of the file.
Validates first 0xA0 bytes on EDAT. This uses appldr with the following execution (see below):
Single execution
Encrypted arguments
CMAC. Key: rifkey, expectedHash: data at 0xA0-0xAF
No Encryption erk:null,riv:null
Validates metadatasections. Again uses appldr:
Multi execution. Blocks upto 0xFC00.
Encrypted arguments
CMAC. Key: rifkey, expectedHash:data at 0x90-0x9F
No Encryption: erk:null, riv:null
Once the NPD is validated (We have validated header upto 0xAF and the metadatasections( which validate the data itself)) an entry on a memory table is created, the entry created by syscall471 is erased and returns. I’ll call this the EDATEncryptionDataTable. This table has 10 entries (remember the limitation of simultaneous files opened??...). Each entry has the following structure:
fileDescriptor: The file descriptor
offset: An additional offset to calculate the position of the metadataSection. Origin unknown. Values seen 0
File len: Length of the uncompressed data. Copied from EDAT offset 0x88-0x8F
flag: Flags required to process the data. Copied from EDAT offset 0x80-0x83. Some flags are:
0x80000000: Debug
0x01000000: SDAT… for those rifkey is calculated differently. Algorithm is not tested yet.
0x00000004: Arguments for appldr are encrypted
0x00000001: Data compressed. MetadataSection length will be 0x20 bytes.
blockSize: The data is stored on blocks of that length. Required for decryption. Copied from EDAT offset 0x84-0x87
numBlocks: Number of blocks on file. Calculated using: numBlocks = (fileLen + blockSize - 1)/blockSize
blockBase: Common part of the key used on decryption. First 0xC bytes of headerHash. Copied from EDAT offset 0x60-0x6B.
devklic: Our rif key. Copied from the entry created by syscall471. FOR SDAT it is calculated using other algorithm.
padding: the digest from NPD. Will be the IV. Copied from EDAT offset 0x40-0x4F.
When an read request is received:
The system determines the blocks required (E.g; read from 0x3FFF to 0x8001 will read blocks 0,1 and 2).
For each of the blocks:
Calculate the blockKey as blockBase concat blockNumber.
Calculate block offset: Start of NPD + offset + 0x100 + metadataSectionLen* numBlocks + blockSize*blockNumber. metadataSectionLen is 0x10 for flag 0xC (if compressed or with special hash would be 0x20).
As you know the appldr is used for decrypting SELFs (that part is documented on the wiki). However there is a second mode that is used for EDAT. To use that mode spu_arg.field35 is 5. For values 0 to 4 it will process a SELF and for others will return error. Graf_chokolo described the structure of spu_args for the SELF. However this changes for EDATs…
Above I indicated several parameters for the appldr. Those parameters are controlled by hashFlag and encryptionFlag.
HashFlag: The hash is performed on input data not decrypted data. So if hash fails nothing is decrypted
EncryptionFlag:
When a key must be decrypted the algorithm is AESCBC128Decrypt.
Keys SHA1:
The multiple execution /single execution refers to the ability of appLdr to store an encrypted (and hashed) data on main memory to resume the hashing/decryption operations on next execution. That data is stored at stateAddress. Basically the functioning has several submodes. One for initialization (that decrypts erk, riv and hkey) and copies the state to main memory, another for processing data, which retrieves the state, process the data and updates the state, and finally another that receives the expected hash and validates. Another submode does all these ops on a single execution (calls the three submodes).
Summing up (for 0xC) for each block:
Calculate data offset: 0x100 + offset(0) + metadataSectionLen*numberOfBlocks + currentBlockNumber*blockSize
Generate block key: concat first 0xC byte of headerHash with current block number
Encrypt block key with rifKey using AESECB128Encrypt. This would be erk and hkey
Calculate riv. It is NPD digest field
Decrypt erk using AESCBC128Decrypt with EDATKEY and RIVKEY -> call result decryptionKey.
Using AESCBC128Decrypt decrypt data from offset to offset + blockSize( if last block calculate remaining bytes rounded up to 0x10 multiple). Key is decryptionKey iv is riv
After searching for those missing bytes I did not found any reference on code. So I zeroed them and feed the file to the PS3... The file was properly loaded and decrypted so:-Those bits are a random padding or-They are not checked ,nor required for creating an EDAT.
That means that you are able to 'free' your EDATs using the method above.Remember that you're going to need the devklic and the rifkey
@Octopus For p3T you don't require the devklic cause for paid decryption you only need the key from rif... so decrypt it and use it directly like any other theme
@EXE.trim.ALL You're right on version. LV2 uses that value to check which flags are valid. However unk4 (and unk3) which are at 0x70 are zeroes. Your info is for the next 16 bytes located at 0x80. Also you separate finalize and type. From the assembly I can say that the original source code considers it a signed integer (int32_t) (that or they have a really weird compiler cause it always load that field as a word (PPC can read a single byte)).
BTW the text on flowers says something about 'werewolves' and the devklicis written as an ASCII text (not binary). It was the first non free content that I decrypted on PC.
It's a pity that I would never enter on efnet (nor other sites) as long as they don't allow proxy and TOR.
notes:
actually wiki accepts proxy (and tor?)
in case you need tor for ef.net, irc.paraphysics.net accepts tor connections
Finally a bit of drama: Hotz8611, I reversed your reactPSN to see the your method and discovered that you just used the info on this post. I haven't check the vsh.self mode but I suppose that youre nullifyng the curve check and generate RAP by encrypting the rifkey (An AES and another crypto algorithm of 5 rounds with shuffle, acummulation and xoring.). I thank you for making people provide me all those RAP than I'm able to transform to rifkey but next time give proper credits (I don't know if your works is based on mallory or mine but obviously is based on info in this topic).
First of all, here is an implementation of the algorithm. It is not fully tested (for example with ISO.BIN.EDAT it validates but fails when decrypting) and is missing the decompression algorithm (I tried deflate but is does not work if someone identifies the algorithm please post it. Blocks start with 0x05). Also keys have been eliminated. On previous port you have the SHA1.
About compression: Instead of having metadatasections of 0x10 byes the new section is 0x20 bytes long.
@jester You probably forgot to decrypt the key. The result of syscall471 must be decrypted using EDATKEY and RIVKEY.
@Octopus to make free files.- Create a memory image of the file.- Decrypt the data, so you have a memory copy of the file decrypted (for compressed no need to decompress)- Modify NPD to make if free (0x03 instead of 0x02). Recalculate hashes using devlikc (that why we need it).- Recrypt each block using the devklic as base to calculate blocks keys (wich will then be transformed again as does the appldr for that type).- Once recrypted, recalculate the hashes of the sections so they matched the new value for the encrypted data.-If compressed recalculate the offset and len on the metadatasection-Using the new metadatasection recalculate data 0x90-0x9F.-Using the new NPD header + new metadatasectionHash calculate hash and place it at 0xA0-0xAF.-Write file to disk (Remember, that file is function of filename, you must overwrite (not recommend while testing) or remember to rename it properly).
The code I posted already implements SDAT. You just need the SDATKEY.The SHA1 is ED2A015EEB1BD0CE06D0447F1A22AF4C1C401E4A
However you won't find it by bruteforce as it is coded as a series of inmediate values. If you check routine sub_5529C at graf_chokolo's dump_lv2 you'll see the routine that checks the edat/sdat header. Few lines below at address 5543C you'll see those inmediate values that are xored with the hash of the NPD to create the fake rifkey (the first 0x100 bytes of the npd are loaded starting at sp + 0x110)
About compression: We don´t know the algorithm but if we decrypt the data and modify the NPD so it looks like a debug you can use make_edata_npdrm to decompress it
Added support for versions 0 and 1 (ISO.BIN.EDAT). See below
When version is 0 or 1 instead of using the digest and the hash of the NPD for calculating the block key and the apploader's IV a zeroed byte array is used as base
I hope this helps your project Snowy.
@dsadsadsa I don't think I could help Kakaroto cause most of the checks are not on current code (They probably added further checking on later firmware version) and my expertise is reading assembly which I don't have. However if I can help then in any form they only need to contact me. In fact the only think that I know that is not already public is that 0x20 bytes are copied from the appldr to main memory just before setting mailbox to 7. Those bytes are at 0x890 and probably is the hash for the whitelist.
@EXE.trim.ALL There is no hash for SDAT. Those values are unknow and not checked(not confirmed). The only function as you have seen is to be xored to generate the key. To get the exacts values I'll need to check the SDK as those are generated there.
@Octopus I haven't look for the second part of the act.dat. My hyphotesis is that the second part of the file is used when debug is enabled and is common for all the consoles (will explain the COD trick). About the digest as I said there is no checked (is a hash of the original data which is unknown until the whole file is read so it can not be used as check). Geohot zeroed it on his code, An analysis of make package will obtain then (or wait until Kakaroto has the SELF fix and hope that it included the algorithm)
About PS2 classic and the need of edat as paid content.
JuanNadie
Flatz
On the .ENC files (AKA PS2 remastered games on PSN):
http://www.ps3hax.net/showthread.php?t=53444 (and the source code) for some missing pieces of EDATA structure
Ada L0ve Lace
Might also means than by forgering your ps2 game to match an original rif purchased on cfw and transfer it on ofw will not work.Actually I think you can buy one game from the store and use its klicensee to encrypt disc content of another game, then you should join the header with ECDSA signature from the original game and encrypted disc content from the another game. But you should carefully choose a boilerplate game because its size should be bigger than size of the game you want to reencrypt, so you can append padding bytes and generate the corrected LIMG segment. I think it should work on OFW too because the ECDSA signature covers the header only and not the data and header stays unmodified with the method above.
Wikify test/Note/[edit]
Flatz:
Refer to my post at http://www.ps3hax.net/showthread.php?t=53444 (and the source code) for some missing pieces of EDATA structure. (Same link as above for PS2 edat but need to be digested complete)
Some naming on description and note on main page are for wikify purpose, not yet finished, could be better/or should be confirmed.
EDAT compressed: can be resigned as free with compressed (obviously) or plain data.
Other stuff
dev_bdvd/PS3_GAME/USRDIR/sce_system.sdat
Path found in game_ext_plugin.sprx. Note is a bluray disc path, and seems to be a 'file reserved name'
Retrieved from ‘http://www.psdevwiki.com/ps3/index.php?title=Talk:EDAT_files&oldid=27050’