Githawk: Alternatives to Buddybuild?

For another self-hosted option, TeamCity(https://www.jetbrains.com/teamcity)?

Usually, CircleCI job queue for open source projects is much less crowded than TravisCI.

My $0.02 from managing some of the RxSwiftCommunity repos.

Travis is absolute garbage (or has become over time).
Queueing is intolerable and slows down development efforts (waiting 50 minutes for a 90s build is unacceptable) and configuration is relatively annoying.

We're slowly but surely moving most repos away to CirlceCI and have been very happy with it. Queueing is really snappy and fair, and the configuration is relatively a breeze.

Also heard great things about Bitrise in that sense.

Super shame about BB, be interesting to keep an eye on it as I don't think Apple would kill it - but yea

Had experience with an older version of Jenkins, as a tool it's definitely capable but would require quite a lot of maintenance/configuration and in the art of keeping things as open and collaborative probably not the best thing honestly

Also if we go for another "big" provider, we run the same risk of it just being swept up by another firm playing Monopoly

I'd be inclined to try Bitrise, my 2 cents

I’m obviously biased ;) but something like this is not gonna happen with App Center.
Please get in touch in case you’re interested.

cc @Palleas

_{Sent with GitHawk}

Https://Buildozer.io supports both iOS and Android. (disclosure: I’m one of its founders)

We've been using CircleCI on all Artsy iOS projects that require a mac for CI - the OSS queues haven't been an issue like travis'

I spent the whole day trying out Bitrise and App Center. So far, they don't look as easy to use and magical as BuddyBuild did...
I'm happy for the BB team (and proud as I'm in Vancouver as well) but very pissed as a user...
BuddyBuild was one of those services that just work, with almost no configuration needed.

I really liked the way how Buddybuild works. I have tired Circle CI before but there are some things to notice here is it uses Fastlane for signing and deploy to Test Flight, Automatic Signing cant be used. have to use "Manual"

I setup some time to chat w/ @TroubleMakerBen, AppCenter _looks_ really nice. Gonna learn more about it and report back!

Check https://buildkite.com/ they offer free account for OSS

Huge fan of bitrise here. We use it a lot with our solution https://www.appaloosa-store.com/

@sregg what exactly wasn't working for you in App Center? I'd be happy to help with anything Build specific.

@derpixeldan for instance, the Slack integration is all manual using webhooks compared to just a checkbox with BB. Also, there's no way to start a build number from a specified number (i.e. my current build number on BB). Finally, signing the iOS app didn't look as straightforward as BB's (I think I only gave them my Apple id username/password and they managed the certificate and provisioning automatically)

I am the engineering manager at Microsoft for App Center Build. We have a great team making improvements every week and we are committed to multi-platform support.

@sregg great feedback and I think improvements in all those areas are on our backlog.

Also feel free to DM me on Twitter at https://twitter.com/0xlukekim with any issues/concerns/feedback.

I have to say I've been very impressed with App Center mainly in the "meta" sense of:

• The development efforts Microsoft are putting into it.
• The ownership and care some of the employees show by commenting here.
• I've seen it in action in AltConf last year and it did seem pretty nice.

Don't have enough experience with it, but seems like a worthy competitor as well :)

Hi everyone, I'm Viktor from https://www.bitrise.io (CTO & Co-founder).

Thank you everyone for the recommendation, it means a lot to our team!

Just wanted to say hi 👋 and ensure you that we're of course listening, feel free to ping us any time via any of our support channels and I'm happy to answer any questions you have here as well!

Haha, I think we can start the bidding war now 😄

My GitHawk brings all the CIs to the yard 🤓

There is also https://buddy.works I didn't use their service, so hard to say if they are good. They have a cool name definitely ;P

I moved over from BuddyBuild to Bitrise (we really want one place for our iOS and Android). It did take some reading of the docs and also some of the step's git repos, but it went pretty smoothly, took about a day alongside doing other things.

@sregg Just to mention we use the "Adjust BuildNumber" step to += 640, as we use it for versionCode and were auto-deploying from BB.

The key differences I found with Bitrise over BuddyBuild (aside from pretty much everything being GUI-based in BB) was to split up the Gradle build into multiple steps. BuddyBuild would (possibly more efficiently) build everything you'd ask for and then pull out the relevant stuff for say deployment emails or publishing to Play Store, with Bitrise you have a couple of choices that I could see: 1. split it up into multiple Build/Gradle steps, e.g. for UI/Android Tests, for your test builds [2x3=6 variants for us], one for your App/Play Store deployment artefacts, with some cleanup steps between (e.g. I change the deploy folder after to prevent emails going out where filters didn't suffice)... or 2. be comfortable with bash script and have a script step that splits up the pipe-map ENV variables so they can be used more easily in later steps.

It would also be nice to have more examples, setting the slack messages to include something like the ones that BB sent by default for e.g. it's nice to be able to customise but generally we just want to be back to writing code.

The other would be features we weren't using, tester management (we use Play Store Alpha and TestFlight), and shake for crash / crash logger (we prefer Firebase).

One of the features that is really cool is being able to not just view and edit your workflow as a .yml file, but download and run that locally with the CLI.

To be honest it's a lot more work than we were used to, that's the point in paying monthly right? But it's pretty much a one-time thing and the bonus is extra customisability. All in all though it's doing the job very well, and for a good price. I'm happy with this switch. I'm sure CircleCI is good too (we use that for our back-end).

It's also worth noting, bitrise's entire infra is OSS - https://github.com/bitrise-io/bitrise.io

Thanks @richardleggett for the feedback, I'll discuss this with the team!

Especially the Slack one - I think that's long overdue now to have a "fancier" (more useful) default message there, instead of requiring you to create your "dream" msg right after dropping in the step the first time. Flexibility is important, but so is the default value / setup experience (and speed). You can tweak the message after that anyway, so there's no issue with making the default more verbose.

Gradle: will discuss it with the tooling team as well, thanks for highlighting!

I've been kicking the can on this (slowly increasing my anxiety). Wanted to link to this blog post exploring alternatives in case there's any more info there.

I want to spend some time w/ @orta and @krausefx while in town to discuss my "north star" vision for automating this project (beyond just CI). Will report back once I muster the energy to actually work on this.

Thanks for the update on this @rnystrom. I feel you. 😕

@rnystrom thanks for the blog post and sorry to hear you had troubles with setting up the distribution on bitrise. Not sure if you saw it but we now have built in Auto Provisioning for code signing, which once configured can manage the the iOS signing files for you automatically: https://blog.bitrise.io/ios-auto-provision-step

Anyway, just wanted to say thanks for trying out bitrise, and to let you know we're always happy to help, in case you'd give another try to bitrise. Feel free to ping me anywhere, e.g. on our Slack (http://chat.bitrise.io).

@viktorbenei For what it's worth, that isn't Ryan's blog post!

Ouch, my bad, it's still quite early in the morning here 😅 Sorry gents and thanks @Sherlouk !

Actually kicked off a Bitrise job last night. Will report back!

_{Sent with GitHawk}

Bitrise build is green! Just as easy to setup as BB. I think we have a winner.

_{Sent with GitHawk}

Glad to hear @rnystrom ! :)

Indeed, initial setup should be quite smooth, similar to BB. The main difference is the configuration UI after that. BB's approach was to provide a simple UI, which implies that certain things might not be possible / can not be changed, while we focused mainly on flexibility, to let you specify every aspect of the process if you like (but this comes with a certain complexity & steeper learning curve). We know that this learning curve might be too much especially for hobby projects and we're working on improving it; quite a few things planned for this year to make deploy etc. configs easier ;)

https://appcenter.ms seems promising

In the name of transparency, here's where we are at the moment: I have both Bitrise and App Center CI building GitHawk. Both services are pretty easy to use, so I want to try out using both to deliver several beta builds and a single App Store build, documenting my process.

Here are my initial thoughts

Bitrise

Pros

• Great support (h/t @viktorbenei)
• Pretty quick builds
• Deploy via fastlane
• Extreme customization & granularity of build steps
• Platform is open source (ish)
• Automatically push builds to ITC from master (I _love_ this)

Cons

• No open source free plan (_yet_)
• Startup (could be acquired or disappear)

App Center

Pros

• Builds are _very_ fast
• Less customization = more streamlining
• Tailored for iOS/Android deployment
• @TroubleMakerBen rocks 😄
• Backed by Microsoft
  
  
  
  • Probably not going away soon
  
  
  • ++resources
  
  

Cons

• No open source free plan (_yet_)
• Requires a shared target to build
• Automated deployment tbd (confirmed its coming)
• Too much stuff we wont use (e.g. I don't need the App Center SDK)
• Log output is _very verbose_, hard to find build errors
• No GitHub status integration

Thanks for sharing @rnystrom ! Just one correction, the web service component of bitrise is not open source, so it's not possible to self-host the API & web UI (yet ;)). All the tools used for running the configuration (the workflow editor, the runner CLI, ...) are open source, so you can download the build config and run it on your own Mac (or on any Mac/Linux), similar to fastlane.

Just a question, for the sake of comparison

Bitrise: Cons: No open source free plan (yet)

Does AppCenter have an open source plan? Might have missed it, AFAIK they don't have one either. I'm really just curious as couldn't find anything related on appcenter's website.

No GitHub status integration

That's a big ☹️

@viktorbenei will update! Not yet either

_{Sent with GitHawk}

Hey folks,

Thx for the feedback and the comparison. I like that and our PMs are looking at this thread.

Automated deployment tbd (confirmed its coming)

We're currently working hard on making Distribution better. Stay tuned.

Does AppCenter have an open source plan? Might have missed it, AFAIK they don't have one either. I'm really just curious as couldn't find anything related on appcenter's website.

We don't have an OSS plan, yet.

Cheers everyone and have a good weekend!

This might be relevant now https://github.com/fastlane/ci 👍

@KrauseFx I've seen this and I'm so stoked about it.

Why ask for features if us as a community can build them ? Plus, Google backing this? couldn't ask for much more.

Really looking forward to contributing code to this as it progresses and applying it to our workflow as it matures.

Thanks for everything you guys are doing for the community !

@KrauseFx player 3 has entered the game

_{Sent with GitHawk}

lolz

I don’t think app center supports per commit [ci skip] like syntax

@dkhamsing unfortunately not (yet).

One real plus for buddybuild that I loved was the way they show before/after/diffs in the test results if an FBSnapshotTestCase unit test produced an unexpected image.

Does anyone know if any of these other systems have a similar feature?

Does App Center support building from a pull request?? I’m very confused cc @TroubleMakerBen

@dkhamsing it does!

edit: never mind 🙊

_{Sent with GitHawk}

@dkhamsing App Center supports build on PUSH, but not build on PR (build on merge), yet.

Ah I see. Thx Ryan, Ben 😊

_{Sent with GitHawk}

After reading this thread it does seem to be a two-horse race: Bitrise and App Center. Yet nobody has touched on the subject of UI tests: I loved how in BB, with just a few clicks, you could run espresso tests on a virtual device. Does any of the two platforms under discussion support this?

https://docs.microsoft.com/en-us/appcenter/test-cloud/

_{Sent with GitHawk}

@dkhamsing App Center Test actually runs UI Tests on physical devices – we have a few thousand. Nope, you can't see pictures ;)

We're actually trying to identify a new CI solution.

• AppCenter: similar to bb but doesn't provide PR support, I think is more focused to management people, also the logs doesn't provide a stack if any task fails.

• Bitrise: Very configurable, offers many open "steps" like codecoverage, deploy, sign, unitTest, UITest, build, deliver, clean and custom, cuz you have the ability to configure it, just got a little confused with .yaml files, you can trigger steps given Push, PR, etc

• Nevercode Very configurable, you can select between gradlew task per branch, build PR, no free plan.

I think at least Bitrise offers many features that we can exploit!

From the link posted above about testing in App Center

1. Review the core concepts
   Understanding the core concepts of the Test Cloud experience improve ease of use, navigation, and communications with support. It is recommended to become familiar with these concepts before running your first tests.

What the hell... I don't want to review any concepts, core or otherwise, I just want it to work with 2 clicks like it did on BB :( I don't have 10 hours to sink into making this work, I'm a coder not a devops engineer...

Yea the App Center documentation could be streamlined

_{Sent with GitHawk}

@acristescu

After reading this thread it does seem to be a two-horse race: Bitrise and App Center. Yet nobody has touched on the subject of UI tests: I loved how in BB, with just a few clicks, you could run espresso tests on a virtual device. Does any of the two platforms under discussion support this?

Not (yet) a single click but also more powerful in a couple of ways: https://blog.bitrise.io/introducing-solid-and-snappy-virtual-device-testing-for-android

We're working on making the setup easier (that's why it's still "beta", not because of lacking functionality ;)).

@acristescu @dkhamsing We are aware! Keep the feedback coming.

@viktorbenei I'm going to try but oh my god that art style is off-putting. I remembered why I stayed away from Bitrise in the past... smiling clouds hugging each other? Sharks with lasers strapped to them?!? Green buttons over a vibrant purple background? Perhaps I'm just showing my age here, but how could I recommend this tool to a client that is... say... a bank?

No problem @acristescu , I definitely see your point, honest feedback is always welcome and a design update is already in the works ;)

I decided to try them both with a simple repo ( https://github.com/acristescu/GreenfieldTemplate ) and see where I get to. So far I've tried App Center and hit a few snags:

• solved (it couldn't find the gradle build tools!), you have to manually add the google() repository to your project's gradle
• it restarted the build number from 1 whereas I already released on play store a 42, google is just going to reject the build!)
• I couldn't find how to test on a virtual device for free, only on real devices with a 30 day free trial.
• I'm not sure it ran the unit tests because

##[warning]No test result files matching /Users/vsts/agent/2.127.0/work/1/s/**/build/test-results/TEST-*.xml were found, so publishing JUnit test results is being skipped.

Not sure what's that about...

Bitrise: On the plus side the setup was just as painless although I think that if I need to change anything I need to bring up an yml file and fiddle with it (update: found a thing called workflow editor. it looks scary but powerful). Snags:

• it never asked me which variant to build and picked the wrong one. I wanted prodRelease but for whatever reason it decided to build exactly the other two mockDebug and prodDebug. Can't find where to change that, but I'm pretty sure there must be one.
• build took longer, 4 mins instead of 2 mins 16 for app center. This is perhaps because of the above problem?
• don't see any mention of junit tests anywhere in the logs. I doubt it ran them. Not obvious how to add them, perhaps somewhere in workflow editor? (update tinkered with the Workflow Editor for about 10 minutes and found it. Bonus points for allowing me the liberty to pick which test target to run)
• not sure what build ID it used, how do I even see that?

Thx for posting these

_{Sent with GitHawk}

Thanks @acristescu for the detailed feedback, we greatly appreciate it. Especially on the warning for JUnit test report files in App Center, this is not affecting your actual test run and should be fixed soon.
Keep it coming!

It took me two hours but I managed to convince app center to upload to google play. However, I cannot seem to convince it to do this automatically, I had to download the signed APK from the app center and then upload it back up into the deploy/stores section (!) to make it work. Seems awefully convoluted, what am I doing wrong?

PS: To add insult to injury, BuddyBuild deployed several times in the same timespan since I forgot to disable it in the beginning and it just works automatically without any human intervention...

Hi @acristescu ,

Re: https://github.com/rnystrom/GitHawk/issues/1330#issuecomment-368228417

it never asked me which variant to build and picked the wrong one. I wanted prodRelease but for whatever reason it decided to build exactly the other two mockDebug and prodDebug. Can't find where to change that, but I'm pretty sure there must be one.

Indeed our current scanner will add a Gradle Runner step, with assembleDebug configured for the base workflow. We realise that this might not be straightforward enough, but in short if you want to build prodRelease then the gradle task is assembleProdRelease. If you want to run lint then the gradle task is lint. You can do all of these with the Gradle Runner step, in fact gradle can handle multiple tasks so to run lint and then assembleProdRelease you can also specify this as the task: lint assembleProdRelease which will do both.

We're working on new steps & new scanner default configs which will make this easier, with more specific steps (e.g. a "Lint" step which runs the gradle lint task, instead of requiring you to set that task in the "Gradle Runner" step) 😉

build took longer, 4 mins instead of 2 mins 16 for app center. This is perhaps because of the above problem?

Indeed that seems to be the case, as assembleDebug most likely generates 2 separate APKs / variants in your case instead of the single "ProdRelease" one.

don't see any mention of junit tests anywhere in the logs. I doubt it ran them.

Specify test as the gradle task input of the Gradle Runner step, that will run your tests - or add the Gradle Unit Test step which is configured to run that gradle task by default.

not sure what build ID it used, how do I even see that?

If you mean whether we set the build number to the bitrise.io build number: by default we don't, you can do that by adding the Change Android versionCode and versionName step for example.

Thanks again for your feedback, we're listening & already scheduled to improve these points of the setup experience! ;)

Awesome discussion. I've been finding it hard to find a BuddyBuild alternative that supports Carthage.

I looked into Nevercode, they only support cocoapods.

I believe the App center supports Carthage.

Any others?

@jamesone

I think the best option for you could be Bitrise, they provide a platform like pipelines bitbucket, also you can configure based on your needs using Steps.

Actually we moved from bb to bitrise, we're using Android and iOS and everything is perfect!

Awesome @cbedoy What did you do about the installable builds that buddybuild provides for all your branches? Does Bitrise have an integration OR support for this?

You can trigger workflows (many _steps_) when you push, create a PR or tag.

Also you can schedule buils per branch.

You should check:

https://devcenter.bitrise.io/bitrise-cli/workflows/
https://devcenter.bitrise.io/bitrise-cli/steps/

When you undertand how bitrise works can create worksflows based of what you need, i.e I want a workflow where just running unitTesting if someone create a PR or I want a workflow where build and generate the .ipa when master was tagged.

Bitrise is something like docker images where you can chose a thirdparty _steps_ for run unitTest, CodeCoverage, or Archive and deploy.

Awesome man! Sounds really interesting. I'll look into it.