mirror of
https://github.com/feather-wallet/feather.git
synced 2025-04-16 18:01:54 +00:00
macos: update gen-sdk
This commit is contained in:
tobtoht
parent
7aa051b58b
commit
ab994bfcde
2 changed files with 28 additions and 83 deletions
|
|
@ -6,95 +6,64 @@
|
|||
|
||||
A free Apple Developer Account is required to proceed.
|
||||
|
||||
Our current macOS SDK
|
||||
(`Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz`)
|
||||
can be extracted from
|
||||
[Xcode_12.2.xip](https://download.developer.apple.com/Developer_Tools/Xcode_12.2/Xcode_12.2.xip).
|
||||
Our macOS SDK can be extracted from
|
||||
[Xcode_15.xip](https://download.developer.apple.com/Developer_Tools/Xcode_15/Xcode_15.xip).
|
||||
|
||||
Alternatively, after logging in to your account go to 'Downloads', then 'More'
|
||||
and search for [`Xcode 12.2`](https://developer.apple.com/download/all/?q=Xcode%2012.2).
|
||||
and search for [`Xcode 15`](https://developer.apple.com/download/all/?q=Xcode%2015).
|
||||
|
||||
An Apple ID and cookies enabled for the hostname are needed to download this.
|
||||
|
||||
The `sha256sum` of the downloaded XIP archive should be `28d352f8c14a43d9b8a082ac6338dc173cb153f964c6e8fb6ba389e5be528bd0`.
|
||||
The `sha256sum` of the downloaded XIP archive should be `4daaed2ef2253c9661779fa40bfff50655dc7ec45801aba5a39653e7bcdde48e`.
|
||||
|
||||
After Xcode version 7.x, Apple started shipping the `Xcode.app` in a `.xip`
|
||||
archive. This makes the SDK less-trivial to extract on non-macOS machines. One
|
||||
approach (tested on Debian Buster) is outlined below:
|
||||
To extract the `.xip` on Linux:
|
||||
|
||||
```bash
|
||||
# Install/clone tools needed for extracting Xcode.app
|
||||
apt install cpio
|
||||
git clone https://github.com/bitcoin-core/apple-sdk-tools.git
|
||||
|
||||
# Unpack Xcode_12.2.xip and place the resulting Xcode.app in your current
|
||||
# Unpack the .xip and place the resulting Xcode.app in your current
|
||||
# working directory
|
||||
python3 apple-sdk-tools/extract_xcode.py -f Xcode_12.2.xip | cpio -d -i
|
||||
python3 apple-sdk-tools/extract_xcode.py -f Xcode_15.xip | cpio -d -i
|
||||
```
|
||||
|
||||
On macOS the process is more straightforward:
|
||||
On macOS:
|
||||
|
||||
```bash
|
||||
xip -x Xcode_12.2.xip
|
||||
xip -x Xcode_15.xip
|
||||
```
|
||||
|
||||
### Step 2: Generating `Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz` from `Xcode.app`
|
||||
### Step 2: Generating the SDK tarball from `Xcode.app`
|
||||
|
||||
To generate `Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz`, run
|
||||
the script [`gen-sdk`](./gen-sdk) with the path to `Xcode.app` (extracted in the
|
||||
previous stage) as the first argument.
|
||||
To generate the SDK, run the script [`gen-sdk`](./gen-sdk) with the
|
||||
path to `Xcode.app` (extracted in the previous stage) as the first argument.
|
||||
|
||||
```bash
|
||||
# Generate a Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers.tar.gz from
|
||||
# the supplied Xcode.app
|
||||
./contrib/macdeploy/gen-sdk '/path/to/Xcode.app'
|
||||
```
|
||||
|
||||
The `sha256sum` of the generated TAR.GZ archive should be `332477876917786b26dd7c3fc1665d2c5cdca81c72755e6a9754f308de77d33b`.
|
||||
The generated archive should be: `Xcode-15.0-15A240d-extracted-SDK-with-libcxx-headers.tar.gz`.
|
||||
The `sha256sum` should be `c0c2e7bb92c1fee0c4e9f3a485e4530786732d6c6dd9e9f418c282aa6892f55d`.
|
||||
|
||||
## Deterministic macOS App Notes
|
||||
|
||||
macOS Applications are created in Linux by combining a recent `clang` and the Apple
|
||||
`binutils` (`ld`, `ar`, etc).
|
||||
macOS Applications are created on Linux using a recent LLVM.
|
||||
|
||||
Apple uses `clang` extensively for development and has upstreamed the necessary
|
||||
functionality so that a vanilla clang can take advantage. It supports the use of `-F`,
|
||||
`-target`, `-mmacosx-version-min`, and `-isysroot`, which are all necessary when
|
||||
building for macOS.
|
||||
All builds must target an Apple SDK. These SDKs are free to download, but not redistributable.
|
||||
See the SDK Extraction notes above for how to obtain it.
|
||||
|
||||
Apple's version of `binutils` (called `cctools`) contains lots of functionality missing in the
|
||||
FSF's `binutils`. In addition to extra linker options for frameworks and sysroots, several
|
||||
other tools are needed as well such as `install_name_tool`, `lipo`, and `nmedit`. These
|
||||
do not build under Linux, so they have been patched to do so. The work here was used as
|
||||
a starting point: [mingwandroid/toolchain4](https://github.com/mingwandroid/toolchain4).
|
||||
The Guix build process has been designed to avoid including the SDK's files in Guix's outputs.
|
||||
All interim tarballs are fully deterministic and may be freely redistributed.
|
||||
|
||||
In order to build a working toolchain, the following source packages are needed from
|
||||
Apple: `cctools`, `dyld`, and `ld64`.
|
||||
|
||||
These tools inject timestamps by default, which produce non-deterministic binaries. The
|
||||
`ZERO_AR_DATE` environment variable is used to disable that.
|
||||
|
||||
This version of `cctools` has been patched to use the current version of `clang`'s headers
|
||||
and its `libLTO.so` rather than those from `llvmgcc`, as it was originally done in `toolchain4`.
|
||||
|
||||
To complicate things further, all builds must target an Apple SDK. These SDKs are free to
|
||||
download, but not redistributable. See the SDK Extraction notes above for how to obtain it.
|
||||
|
||||
The Guix process builds 2 sets of files: Linux tools, then Apple binaries which are
|
||||
created using these tools. The build process has been designed to avoid including the
|
||||
SDK's files in Guix's outputs. All interim tarballs are fully deterministic and may be freely
|
||||
redistributed.
|
||||
|
||||
As of OS X 10.9 Mavericks, using an Apple-blessed key to sign binaries is a requirement in
|
||||
order to satisfy the new Gatekeeper requirements. Because this private key cannot be
|
||||
shared, we'll have to be a bit creative in order for the build process to remain somewhat
|
||||
deterministic. Here's how it works:
|
||||
Using an Apple-blessed key to sign binaries is a requirement to produce (distributable) macOS
|
||||
binaries. Because this private key cannot be shared, we'll have to be a bit creative in order
|
||||
for the build process to remain somewhat deterministic. Here's how it works:
|
||||
|
||||
- Builders use Guix to create an unsigned release. This outputs an unsigned ZIP which
|
||||
users may choose to bless and run. It also outputs an unsigned app structure in the form
|
||||
of a tarball, which also contains all of the tools that have been previously (deterministically)
|
||||
built in order to create a final DMG.
|
||||
users may choose to bless, self-codesign, and run. It also outputs an unsigned app structure
|
||||
in the form of a tarball.
|
||||
- The Apple keyholder uses this unsigned app to create a detached signature, using the
|
||||
script that is also included there. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs).
|
||||
- Builders feed the unsigned app + detached signature back into Guix. It uses the
|
||||
pre-built tools to recombine the pieces into a deterministic ZIP.
|
||||
included script. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs).
|
||||
- Builders feed the unsigned app + detached signature back into Guix, which combines the
|
||||
pieces into a deterministic ZIP.
|
||||
|
|
|
|||
|
|
@ -8,21 +8,6 @@ import gzip
|
|||
import os
|
||||
import contextlib
|
||||
|
||||
# monkey-patch Python 3.8 and older to fix wrong TAR header handling
|
||||
# see https://github.com/bitcoin/bitcoin/pull/24534
|
||||
# and https://github.com/python/cpython/pull/18080 for more info
|
||||
if sys.version_info < (3, 9):
|
||||
_old_create_header = tarfile.TarInfo._create_header
|
||||
def _create_header(info, format, encoding, errors):
|
||||
buf = _old_create_header(info, format, encoding, errors)
|
||||
# replace devmajor/devminor with binary zeroes
|
||||
buf = buf[:329] + bytes(16) + buf[345:]
|
||||
# recompute checksum
|
||||
chksum = tarfile.calc_chksums(buf)[0]
|
||||
buf = buf[:-364] + bytes("%06o\0" % chksum, "ascii") + buf[-357:]
|
||||
return buf
|
||||
tarfile.TarInfo._create_header = staticmethod(_create_header)
|
||||
|
||||
@contextlib.contextmanager
|
||||
def cd(path):
|
||||
"""Context manager that restores PWD even if an exception was raised."""
|
||||
|
|
@ -60,13 +45,8 @@ def run():
|
|||
sdk_build_id = pl['ProductBuildVersion']
|
||||
print("Found MacOSX SDK (version: {sdk_version}, build id: {sdk_build_id})".format(sdk_version=sdk_version, sdk_build_id=sdk_build_id))
|
||||
|
||||
clang_rt = xcode_app.joinpath("Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/lib/darwin")
|
||||
|
||||
out_name = "Xcode-{xcode_version}-{xcode_build_id}-extracted-SDK-with-libcxx-headers".format(xcode_version=xcode_version, xcode_build_id=xcode_build_id)
|
||||
|
||||
xcode_libcxx_dir = xcode_app.joinpath("Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1")
|
||||
assert xcode_libcxx_dir.is_dir()
|
||||
|
||||
if args.out_sdktgz:
|
||||
out_sdktgz_path = pathlib.Path(args.out_sdktgz_path)
|
||||
else:
|
||||
|
|
@ -74,7 +54,7 @@ def run():
|
|||
out_sdktgz_path = pathlib.Path("./{}.tar.gz".format(out_name))
|
||||
|
||||
def tarfp_add_with_base_change(tarfp, dir_to_add, alt_base_dir):
|
||||
"""Add all files in dir_to_add to tarfp, but prepent MEMBERPREFIX to the files'
|
||||
"""Add all files in dir_to_add to tarfp, but prepent alt_base_dir to the files'
|
||||
names
|
||||
|
||||
e.g. if the only file under /root/bazdir is /root/bazdir/qux, invoking:
|
||||
|
|
@ -109,10 +89,6 @@ def run():
|
|||
with tarfile.open(mode="w", fileobj=gzf, format=tarfile.GNU_FORMAT) as tarfp:
|
||||
print("Adding MacOSX SDK {} files...".format(sdk_version))
|
||||
tarfp_add_with_base_change(tarfp, sdk_dir, out_name)
|
||||
print("Adding libc++ headers...")
|
||||
tarfp_add_with_base_change(tarfp, xcode_libcxx_dir, "{}/usr/include/c++/v1".format(out_name))
|
||||
print("Adding clang_rt libraries")
|
||||
tarfp_add_with_base_change(tarfp, clang_rt, "{}/lib/darwin".format(out_name))
|
||||
print("Done! Find the resulting gzipped tarball at:")
|
||||
print(out_sdktgz_path.resolve())
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue