Software Update Service - Revision history

J. J.: Made it more serious.

2013-07-31T20:56:12Z

Made it more serious.

5urd: /* BOM Framework */

2012-04-06T00:26:26Z

‎BOM Framework

OMEGA RAZER: /* Tasks: Applying updates on a jailbreaked system */

2009-09-14T05:04:17Z

‎Tasks: Applying updates on a jailbreaked system

ChronicDev: /* Sample patch checker */

2008-11-01T16:32:32Z

‎Sample patch checker

Pumpkin: Not relevant to article itself. "Do not delete" doesn't make sense, as this wiki keeps all histories, etc. The preamble is more suited to the talk page.

2008-07-30T16:15:51Z

Not relevant to article itself. "Do not delete" doesn't make sense, as this wiki keeps all histories, etc. The preamble is more suited to the talk page.

ChronicDev at 20:13, 29 July 2008

2008-07-29T20:13:21Z

Show changes

ChronicDev: New page: == Introduction == This is from the old iPhone Dev Wiki, but is not cached or saved anywhere else. I saw how in-depth it got and at the time I was thinking that it would get removed. I was...

2008-07-29T20:05:32Z

New page: == Introduction == This is from the old iPhone Dev Wiki, but is not cached or saved anywhere else. I saw how in-depth it got and at the time I was thinking that it would get removed. I was...

New page

== Introduction ==
This is from the old iPhone Dev Wiki, but is not cached or saved anywhere else. I saw how in-depth it got and at the time I was thinking that it would get removed. I was right. Luckily, I saved it, and I have been holding on to it for when the time was right. Now that we have this open wiki, the time is right. This article details the old software updates, usually less than 10mb in size, that Apple used to update firmwares. They worked much better, and were based on good old .patch files. Please do not delete this preamble. And without further ado, here is the article.

PS: I have slightly editted it because of formatting reasons, as well as made it look neater.

== Tasks: Applying updates on a jailbreaked system ==

Describing the update process: DONE

Understanding the BOM file format on the iPhone: IN PROGRESS (see the BOM framework)

Patch the pre/post BOM to only check the files that are patched: IN PROGRESS (compare BOM list with cpio'ed archive content)

Regenerating/signing the update files: IN PROGRESS (iZsh has some stuff)

== Update archive content ==

The update archive is a zip contains the following files.

archive.cpio.gz: the actual cpio patch archive (encrypted). Contains a list of BSDiff patches (*) and baseband firmware updates, if available, with the associated flashing tools (bbupdater/imeisv).

Info.plist: description of the update and hashes of the update components (cleartext)

Info.plist.signature: asymmetric signature of Info.plist validated against /System/Library/Lockdown/iPhoneSoftwareUpdate.pem

libupdate_brain.dylib: stage 2 update process library (encrypted)

pre.bom: Bill Of Material stating the filesystem state before the update process (encrypted)

post.bom: Bill Of Material stating the filesystem state after the update process (encrypted)

(*) there is a different update file to go from 1.0 to 1.0.2 and from 1.0.1 to 1.0.2 since bsdiff does not apply partial patches.

The BOM files can be used to validate the CRC 32 checksums of the modified files. They can be browsed with lsbom on OS X.

== Update process ==

# Software_Update is invoked

# The files signatures are checked

# libupdate_brain.dylib, pre.bost and post.bom are decrypted

# gogo_software_update in libupdate_brain.dylib is called to perform the actual update process
Blocking automatic updates

A quick reversible way to disable automatic updates: remove the executable permission on software_update

chmod a-x /usr/libexec/software_update

Since bbupdater is included in the updates, you must block all automatic updates to prevent baseband updates

== Debugging software_update ==

software_update fails when launched from dropbear because it cannot connect to lockdownd (validated with weasel, the minimal debugger included in the toolchain)

Further investigation is needed to know why this happen. Probably because it must be spawned from a specific process. In this case weasel could probably be modified to accept remote commands in order to debug that kind of processes with minimal side effects.

== Decrypting the update files ==

The update files are AES-128 CBC encrypted with a key dependant of the software version.

To retrieve the key, compute a SHA-1 hash of /System/Library/Caches/com.apple.kernelcaches/kernelcache.s5l8900xrb from offset (size / 2 - 0x2000) for 0x4000 bytes (or used the utility described below)

To decrypt the update files from a desktop computer - funny facts : the IV is TheIphoneLovesU

openssl enc -d -aes-128-cbc -K hex_key_obtained_in_the_previous_step -iv 5468656950686F6E654C6F7665735500 -in encrypted_file -out decrypted_file

Another quick & dirty way to recover the decrypted files: Have software_update crash before running stage 2 and scavenge the files, for example by modifying the name of the function called in stage 2 (gogo_software_update) - the decrypted files are in /private/var/software_update/foo.pkg
Reversing BOM

The BOM framework (located in /System/Library/PrivateFrameworks/Bom.framework/Bom) is not documented, but very easy to reverse.

Here is the .h used to implment the prototype checker

// BOM framework header - v0.1
// Zf for iPhone Developers Wiki
// Licensed under GPLv2
typedef void* BOM;
typedef void* BOMEnumerator;
typedef void* BOMFSObject;
typedef enum BOMObjectType {
BOMFile = 1,
BOMDirectory,
BOMLink
} BOMObjectType;
typedef struct BOMSize {
int low;
int high;
} BOMSize;
int BOMCRC32ForFile(char *path, unsigned int *crc, BOMSize *size);
BOM BOMBomOpen(char *path, int unk1); // 0 == error
// unk1 = 0
int BOMBomFree(BOM bom);
BOM BOMBomNew(char *path);
int BOMBomCommit(BOM bom);
int BOMBomInsertFSObject(BOM bom, BOMFSObject object, int unk1);
// unk1=0

BOMEnumerator BOMBomEnumeratorNewWithOptions(BOM bom, int unk1, int unk2);
// unk1=0 unk2=1
BOMFSObject BOMBomEnumeratorNext(BOMEnumerator enumerator); // 0 == last
int BOMEnumeratorFree(BOMEnumerator enumerator);
BOMObjectType BOMFSObjectType(BOMFSObject object);
char* BOMFSObjectPathName(BOMFSObject object);
unsigned int BOMFSObjectChecksum(BOMFSObject object);
unsigned int BOMFSObjectSize(BOMFSObject object);
int BOMFSObjectFree(BOMFSObject object);

== Sample patch checker ==

The following code (running on the iPhone to validate the checksums or on OS-X to modify the BOM), partially tested, creates a simplified BOM containing only references to the patched files. This does not work fully as expected because during the update process an error is generated if the file system contains more files than what is provided in the BOM. But the idea is still sound :)

Program arguments : pre.bom, directory containing the extract patches (cpio -i <archive.cpio), target.bom to be created and / if running on the iPhone

// Checkpatch - Sample update BOM cleaner - v0.1
// Zf for iPhone Developers Wiki
// Licensed under GPLv2

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "bom.h"

int main(int argc, char **argv) {
BOM src, dst;
BOMEnumerator bomEnumerator;
BOMFSObject bomObject;
char *root_dir = NULL;
int count=0, errors=0;

if (argc < 4) {
printf("Usage: %s src_BOM patches_dir target_BOM [root_dir]\n",
argv[0]);
printf("\nCreate target_BOM a reduced version of src_BOM "
"containing only the references to patches_dir patches."
"\nChecksums are validated against root_dir if given\n");
return 1;
}

src = BOMBomOpen(argv[1], 0);
dst = BOMBomNew(argv[3]);
if (argc >= 4) {
root_dir = argv[4];
}

if (!src) {
printf("Couldn't open %s", argv[1]);
return 1;
}
bomEnumerator = BOMBomEnumeratorNewWithOptions(src, 0, 1);
if (!bomEnumerator) {
printf("Open enumerator failed\n");
return 1;
}
while ((bomObject = BOMBomEnumeratorNext(bomEnumerator))) {
BOMObjectType fsObjectType;
unsigned int expectedCRC;
char found = 0;

fsObjectType = BOMFSObjectType(bomObject);
if (fsObjectType == BOMFile) {
char patchPath[400];
sprintf(patchPath, "%s/%s.patch",
argv[2],
BOMFSObjectPathName(bomObject));
found = (access(patchPath, F_OK) == 0);
}
else
if (fsObjectType == BOMDirectory) {
char patchPath[400];
sprintf(patchPath, "%s/%s", argv[2],
BOMFSObjectPathName(bomObject));
found = (access(patchPath, F_OK) == 0);
}

expectedCRC = BOMFSObjectChecksum(bomObject);
if (found && root_dir != NULL && expectedCRC != 0) {
char rootPath[400];
unsigned int computedCRC;
sprintf(rootPath, "%s/%s", root_dir,
BOMFSObjectPathName(bomObject));
if (access(rootPath, R_OK) != 0) {
errors++;
fprintf(stderr, "%s not found\n",
rootPath);
found = 0;
}
BOMCRC32ForFile(rootPath, &computedCRC, NULL);
if (computedCRC != expectedCRC) {
errors++;
fprintf(stderr, "CRC mismatched for %s "
"expected %u got %u\n",
rootPath, expectedCRC, computedCRC);
found = 0;
}
}

if (found) {
count++;
BOMBomInsertFSObject(dst, bomObject, 0);
}
BOMFSObjectFree(bomObject);
}
BOMBomEnumeratorFree(bomEnumerator);
BOMBomFree(src);

BOMBomCommit(dst);

if (root_dir == NULL) {
printf("%d entries added. No consistency check done\n",
count);
}
else {
if (errors) {
printf("%d entries added, %d errors.\n"
"VALIDATE BEFORE PATCHING !\n",
count, errors);
}
else {
printf("%d entries added, good to go :)\n",
count);
}
}

return 0;
}

== Update utilities ==

A set of utilities to manipulate update files on the iPhone is provided - current version 0.1 available here http://dl.free.fr/aqIyfnGZU/iPhone-update-utilities-0.1.tar.gz

These utilities should be trivial to port to OS X.

Utilities currently provided

bspatch Port of bspatch used to apply specific update patches
extracted from the cpio archive
bspatch [original file] [patch file] [target file]
The original file and target file can be the same

getUpdateKey Retrieve the update key used to decrypt libupdate_brain.dylib
(and subsequent data files : boms and the main archive)
Run without parameters or with the path to a specific
kernelcache

listBOM Very simple lsbom - list the contents of a BOM file

checkBOM Validates BOM checksums against a list of files
checkBOM [BOM file] [path to root]
On the iPhone path to root will be /

reduceBOMPatch Create a reduced BOM limited to files matching a patch set
reduceBOMpatch [source BOM] [patch dir.] [target BOM] [path
to root (optional)]
If a root path is provided the BOM files checksum will be
checked against the local filesystem

== Early alpha: applying an update without wiping modified content ==

The following steps were used to update from 1.0.1 to 1.0.2 without wiping everything. It's still not user friendly, and buggy, but you can try to follow them and report your ideas to improve it

Pre-steps (to be performed once or once per update)

1) Install your own Software Update certificate. This will block Apple automatic updates (interesting side effect :p) but more importantly allow you to install your own.

The file to replace is /System/Library/Lockdown/iPhoneSoftwareUpdate.pem

2) Obtain your current update key according to your kernel cache version

Run getUpdateKey on the iPhone and copy the result

Preparing the update files

1) Obtain your update file from iTunes, do not install it. The file name should be iPhone1,updateVersion_initialBuild_to_updateBuild_Update.ipsw

2) Unzip it

3) Decrypt archive.cpio.gz, libupdate_brain.dylib, pre.bom, post.bom with

openssl enc -d -aes-128-cbc -K update_key -iv 5468656950686F6E654C6F7665735500 -in encrypted_file -out decrypted_file

4) Extract archive.cpio.gz with gzip -d archive.cpio.gz ; cpio -i < archive.cpio

(here you could choose specific patches to install, remove the baseband update, and so on ... but this first version is complicated enough :p)

Generating the modified BOMs

1) Copy the obtaines patches/ directory, pre.bom, post.bom to the iPhone, for example in /tmp/upd

2) Prepare the reduced pre BOM : from /tmp/upd run

reduceBOMPatch pre.bom patches/ pre.bom.reduced /

If you get a checksum warning - fix it (for example reinstall the original lockdownd if it was modified)

3) Prepare the reduced post BOM : from /tmp/upd run

reduceBOMPatch post.bom patches/ post.bom.reduced

4) Upload back pre.bom.reduced and post.bom.reduced to your computer

Patching libupdate_brain.dylib (temporary step, ugly hack, and so on)

Several controls are made on the BOM file that are not patched correctly - local supplementary files should be merged with the BOM in the visited directories.

For the moment patching libupdate_brain.dylib to remove the control (call to verify_uberbom) is faster :)

The following patch is valid for the for the 1.0.2 update and probably below

Comparing files libupdate_brain.dylib and LIBUPDATE_BRAIN.DYLIB.MODIF
000059CC: BE 00
000059CD: F2 00
000059CE: FF A0
000059CF: EB E3
00006670: 95 00
00006671: EF 00
00006672: FF A0
00006673: EB E3

=== Preparing the new update archive ===

1) Rename pre.bom.reduced pre.bom and post.bom.reduced post.bom.

If you did some modifications to the patches, put them back in the gzipped cpio archive
Have the original Info.plist ready as well

2a) Edit Info.plist, set the Encrypted key to <false/> OR process to 2b)

2b) Encrypt pre.bom, post.bom, libupdate_brain.dylib and archive.cpio.gz with the following command

openssl enc -e -aes-128-cbc -K update_key -iv 5468656950686F6E654C6F7665735500 -in cleartext_file -out encrypted_file

3) Edit Info.plist and update the base64 digests for each modified component. You'll obtain the new digest with

openssl sha1 -binary file_to_check | openssl base64 -e

4) Generate Info.plist.signature with your own private key associated to the update certificate you replaced with

cat Info.plist | openssl dgst -sha1 -key /path/to/your/private/key > Info.plist.signature

5) Zip back all files and replace iTunes ipsw

Running the update and checking all is fine

1) Connect to iTunes and install the update. If you're prompted to restore, you lose :p

2) Upload post.bom.reduced to /tmp/upd on the Iphone and run from /tmp/upd

checkBOM post.bom.reduced /

You should not see any checksum error. If you see some, report the problem, it means that some patches have not been applied.

3) If some patches are missing, upload them on the iPhone in /tmp/upd and run

bspatch /file/to/patch /tmp/upd/patch_content /file/to/patch

4) Revalidate with step 2)

If you still see checksum errors, go buy a Nokia :)

== Sample pre BOM list between 1.0 and 1.0.2 ==

See http://www.pastebin.ca/668944

The checksum is the last big number before the date. It's a typical CRC 32 checksum than can be checked with GNU cksum

@@ Line 266: / Line 266: @@
 === Pre-steps (to be performed once or once per update) ===
-# Install your own Software Update certificate. This will block Apple automatic updates (interesting side effect :p) but more importantly allow you to install your own.
+# Install your own Software Update certificate. This will block Apple automatic updates (interesting side effect) but more importantly allow you to install your own.
   The file to replace is /System/Library/Lockdown/iPhoneSoftwareUpdate.pem
@@ Line 286: / Line 286: @@
 # Extract archive.cpio.gz with gzip -d archive.cpio.gz ; cpio -i < archive.cpio
-(here you could choose specific patches to install, remove the baseband update, and so on ... but this first version is complicated enough :P)
+(here you could choose specific patches to install, remove the baseband update, and so on)
 === Generating the modified BOMs ===
@@ Line 346: / Line 346: @@
 === Running the update and checking all is fine ===
-# Connect to iTunes and install the update. If you're prompted to restore, you lose :p
+# Connect to iTunes and install the update. If you're prompted to restore, you have failed.
 # Upload post.bom.reduced to /tmp/upd on the Iphone and run from /tmp/upd

← Older revision		Revision as of 05:04, 14 September 2009
Line 1:		Line 1:
−	== Tasks: Applying updates on a ~~jailbreaked~~ system ==	+	== Tasks: Applying updates on a jailbroken system ==

	<b>Describing the update process</b>: DONE		<b>Describing the update process</b>: DONE

← Older revision		Revision as of 16:15, 30 July 2008
Line 1:		Line 1:
−	== Introduction ==
−	This is from the old iPhone Dev Wiki, but is not cached or saved anywhere else. I saw how in-depth it got and at the time I was thinking that it would get removed. I was right. Luckily, I saved it, and I have been holding on to it for when the time was right. Now that we have this open wiki, the time is right. This article details the old software updates, usually less than 10mb in size, that Apple used to update firmwares. They worked much better, and were based on good old .patch files. <b>Please do not delete this preamble</b>. And without further ado, here is the article.
−
−	PS: I have slightly editted it because of formatting reasons, as well as made it look neater.
−
	== Tasks: Applying updates on a jailbreaked system ==		== Tasks: Applying updates on a jailbreaked system ==

@@ Line 75: / Line 75: @@
 === BOM Framework ===
 The BOM framework (located in /System/Library/PrivateFrameworks/Bom.framework/Bom) is not documented, but very easy to reverse.
  // BOM framework header - v0.1
@@ Line 85: / Line 83: @@
  typedef void* BOMFSObject;
  typedef enum BOMObjectType {
-	BOMFile = 1,
+     BOMFile = 1,
-	BOMDirectory,
+     BOMDirectory,
-	BOMLink
+     BOMLink
  } BOMObjectType;
  typedef struct BOMSize {
-	int low;
+     int low;
-	int high;
+     int high;
  } BOMSize;
  int BOMCRC32ForFile(char *path, unsigned int *crc, BOMSize *size);
@@ Line 101: / Line 99: @@
  int BOMBomInsertFSObject(BOM bom, BOMFSObject object, int  unk1);
  // unk1=0
  BOMEnumerator BOMBomEnumeratorNewWithOptions(BOM bom, int unk1, int unk2);
  // unk1=0 unk2=1

@@ Line 115: / Line 115: @@
 The following code (running on the iPhone to validate the checksums or on OS-X to modify the BOM), partially tested, creates a simplified BOM containing only references to the patched files. This does not work fully as expected because during the update process an error is generated if the file system contains more files than what is provided in the BOM. But the idea is still sound :)
-Program arguments : pre.bom, directory containing the extract patches (cpio -i <archive.cpio), target.bom to be created and / if running on the iPhone
+//Program arguments : pre.bom, directory containing the extract patches (cpio -i <archive.cpio), target.bom to be created and / if running on the iPhone
 // Checkpatch - Sample update BOM cleaner - v0.1
 // Zf for iPhone Developers Wiki
@@ Line 230: / Line 230: @@
 	return 0;
+}
 == Update utilities ==