What is missing to run the iOS tests?

[loes]

I was checking the results of the buildbots, and I can see that the Android tests are running but the iOS tests are not. Would it be possible for the tests to be run and the results to be ignored, so that we could at least see what is broken? Or, what is missing to be able to run these tests?

[alexcrichton]

AFAIK there’s never been effort to even try this, so it very well may work. To get this done the build system would probably need to do something similar to Android:

• Start up an emulator when the build starts
• Update compiletest and rustbuild to ship artifacts to the emulator
• Run tests inside the emulator and collect the results

If all that’s done then we’d gladly start gating on iOS tests!

[comex]

There is no iOS emulator, only the iOS Simulator which is actually just x86.

[loes]

Thanks Alex! I’d be happy to help with this in early Jan. I’ll need to do some studying to help with with any needed changes to the compiler, but I could certainly help with testing deployment to simulator or device.

AFAIK an Apple dev account would be required for deployment to the device, but they’re also free now.

[loes]

Correct, AFAIK any ARM tests would have to be done on the device.

[loes]

Hi Alex,

It seems that a tool called Dinghy already exists, and the authors there did the hard work of figuring out a lot of the steps required. By looking at that and also through various other articles, I was able to come up with the following steps which will compile the tests and run them on an iOS simulator:

---

Prerequisites:

• rustup target install x86_64-apple-ios

Steps:

1. Create a shim to the cross-compiling linker for the simulator:

xcrun --sdk iphonesimulator --show-sdk-path

a) With the given path, create a shim called linker_shim.sh:

  #!/bin/sh
  cc -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator10.2.sdk "$@"

b) Make it executable:

chmod 777 linker_shim.sh

2. Set an environment variable so that cargo doesn’t use the default linker:

export CARGO_TARGET_X86_64_APPLE_IOS_LINKER=~/<path>/linker_shim.sh

3. Compile tests:

cargo test --no-run --target=x86_64-apple-ios

4. Wrap as an app:

a) Copy executable to a separate empty directory.

mkdir app
cp target/x86_64-apple-ios/debug/hello-a254f99d8b1ee1bc app/hello 

b) Create an Info.plist file with the following contents:

  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>CFBundleExecutable</key>
      <string>hello</string>
      <key>CFBundleIdentifier</key>
      <string>com.rust.tests</string>
      <key>UIRequiredDeviceCapabilities</key>
      <array>
        <string>x86_64</string>
      </array>
    </dict>
  </plist>

5. Start the iOS simulator:

open /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app

6. Install app:

xcrun simctl install booted app/

7. Use LLDB to prepare to receive process return value:

   lldb platform select ios-simulator platform connect process attach -n hello --waitfor

8. In another terminal window, run app:

xcrun simctl launch booted com.rust.tests

9. in LLDB, complete execution:

continue

Output will look as follows:

Process 21635 resuming Process 21635 exited with status = 0 (0x00000000)

In the case of a test failure, the output will instead look as follows:

Process 21742 resuming Process 21742 exited with status = 101 (0x00000065)

---

Gist: https://gist.github.com/learnopengles/60c945b92fbe60a0fecaa0143c35c4af

---

I’m not really sure right now how to make this more automated let alone integrate it into the compiler’s test framework, but just wanted to document everything I found so far.

[alexcrichton]

Whoa, awesome!

If you want to edit our .travis.yml to run those tests on Travis, I’d totally be down for doing so!

[loes]

Hi Alex,

I saw that Android was integrated into the compile test system. Would you want to do the same for the iOS simulator or just have it strictly be something contained in a script? I think that could be doable if we can control the compiler unit tests the same as in the example above, and we know where to find the executable (it can have a deterministic name).

[alexcrichton]

Either way is fine by me, I’m down for whatever’s easiest to implement!

[loes]

I have a proof of concept for regular cargo projects that automates all of the above. It runs the tests, prints out the results, and will also manage the iOS simulator instance so that there’s no need to muck around with the GUI: https://gist.github.com/learnopengles/ebf6c04180ca7d81b83e37945b1870cc

This will work for a cargo project (not heavily tested but it should be fine for anything that can compile to x86_64-apple-ios), and also I’ve been in touch with the Dinghy author and cargo dinghy test can also test on the iOS simulator (with this route at the moment you have to manage the instances yourself).

So I suppose the next step will be to adapt this to rustbuild test. Does it work similarly to cargo test? If so it should be straightforward. I’ll take a look at this next.

[alexcrichton]

Awesome work! Yeah rustbuild is relatively similar to cargo test. The two tests methods you’ll be interested in are:

1. We run tests through compiletest (a tool in tree) which runs all the src/test/* suites such as run-pass, compile-fail, etc. You can likely crib a lot of logic from Android here. Each test just compiles locally, ships a binary to the emulator, runs it, and then reports the output.
2. In rustbuild itself we run crate tests. They’re all compiled with cargo test --no-run and then shipped to an emulator in the same way compiletest does it.

Updating those two locations should be all you need!

[loes]

Thanks, that helps out! I’ll get started on this.

[loes]

So I searched for all usages of adb to get an idea of which places I need to modify, and the first part I tried to make the change is in rust/src/vendor/libc/ci/run.sh as that seemed to be a simple place to start. However, it’s failing as there is no libc-test in that folder. Where does that come from?

I tried cloning https://github.com/alexcrichton/ctest but that doesn’t seem to produce a binary.

---

EDIT: Nevermind, I think I see what’s happening here. It’s a vendored dependency and that folder wasn’t included, so I should probably just ignore this?

[alexcrichton]

Ah yeah looks like you’ve discovered that it’s an external repo: https://github.com/rust-lang/libc

[system]

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.