From zero to 10 million strains of Kotlin

• We’re sharing classes discovered from shifting our Android growth from Java to Kotlin.
• Kotlin is a well-liked language for Android growth and affords some key benefits over Java. 
• As of right now, our Android codebase incorporates over 10 million strains of Kotlin code.
• We’re open sourcing numerous examples and utilities we used to govern Kotlin code as a part of this migration

In recent times, Kotlin has turn into a well-liked language for Android growth. So it solely is smart that we’d shift our Android growth at Meta to Kotlin as we work to make our growth workflows extra environment friendly. 

Meta’s Android repository could be very massive and reaches throughout our household of apps and applied sciences, together with Fb, Instagram, Messenger, Portal, and the Quest. Shifting away from Java, which we presently use for Android growth, and over to Kotlin will not be a trivial job.

Why we’re changing our codebase to Kotlin

Kotlin is usually thought to be a greater language than Java, with increased favorability scores than Java within the yearly Stack Overflow developer survey. We additionally in contrast the most recent Kotlin model with Java 11, which is the most recent model that can be utilized for Android growth.

Other than its reputation, Kotlin holds some main benefits:

• Nullability: Null pointer exceptions are a standard drawback at Meta, as in every single place else. We’re superb at fixing them earlier than releasing our apps, however coping with these points remains to be time-consuming. We use inside instruments to detect null issues of safety earlier, and we rigorously annotate our code as a part of our work to detect such points in Java earlier. However even with that, Kotlin’s built-in nullability dealing with is extra strong and simpler to work with.
• Useful programming: Kotlin’s assist for inline features and lambda expressions permits us to make use of a useful programming model with out compromising execution pace. Though Java 8 provides assist for lambdas and is accessible for Android, it comes at the price of extra nameless objects, which have an effect on efficiency negatively on low-end Android units. Meta’s home-brewed Redex minimizes these points, however they nonetheless exist, making Kotlin a greater different.
• Shorter code: Kotlin’s fashionable design makes its code shorter. Kotlin permits for dropping express sorts (as does Java 11), and along with the usual library, which relies on the useful model talked about above, it shortens many repetitive loops into easier statements. This shorter code can also be extra express, which may make it simpler to observe.
• Area-specific language (DSL) / Kind-safe builders: Kotlin’s numerous options come collectively and allow us to outline a DSL. Mainly, a approach to transfer definitions corresponding to Android XMLs to be carried out instantly in Kotlin code. However this device needs to be wielded fastidiously as a result of implementing DSLs in Kotlin can both be helpful or flip into overengineering.

 Nonetheless, adopting Kotlin additionally has a number of disadvantages that we couldn’t ignore:

• Adopting one other language might imply we’ll need to cope with a combined codebase of two languages for a very long time. Kotlin is excellent at interacting with Java, however quirks do pop up at instances.
• Kotlin is a well-liked language, however in contrast with Java, the recognition hole is obvious. Java is the world’s second or third hottest language (relying on how one measures this). This implies fewer instruments can be found. Worse than that, all of the Kotlin instruments must account for Kotlin and Java interoperability, which complicates their implementation.

Lastly, our greatest fear was construct instances. We knew from the beginning that Kotlin’s construct instances can be longer than Java’s. The language and its ecosystem are extra difficult, and Java had 20 years of a head begin to optimize its compiler. Since we personal a number of massive apps, the results of longer construct instances might negatively affect our builders’ expertise. Listening to anecdotes corresponding to OkHttp’s expertise migrating to Kotlin painted a less-than-ideal image.

How we’re approaching the migration

Migrating to Kotlin is each surprisingly simple and really difficult. It’s simple as a result of Kotlin’s design permits easy conversion from Java with well-thought-out interoperability. This design made it attainable for JetBrains to provide the developer neighborhood with J2K, the Java to Kotlin converter that comes with IntelliJ/Android Studio.

However even with J2K, the migration remains to be difficult. J2K doesn’t all the time get issues right, and the interoperability of Java and Kotlin exposes us to a number of edge circumstances. These run the gamut from model fixes to make the code cleaner all the way in which to difficult runtime habits modifications (which we are going to talk about later).

Going into this migration, we had two choices:

1. We might make it attainable to put in writing new code at Meta utilizing Kotlin however depart a lot of the current code in Java.
2. We might try to convert nearly all our in-house code into Kotlin.

The benefit of the primary possibility is obvious — it’s a lot much less work. However there are two notable disadvantages to this method. First, enabling interoperability between Kotlin and Java code introduces using platform sorts in Kotlin. Platform sorts give rise to runtime null pointer dereferences that end in crashes as a substitute of the static security supplied by pure Kotlin code. In some difficult circumstances, Kotlin’s null test elision can let nulls by way of and create stunning null pointer exceptions later. This might occur if, for instance, Kotlin code calls a Kotlin interface carried out by a Java interface.

Different points embody Java’s incapability to tag sort parameters as nullable (till lately), and Kotlin’s overloading guidelines taking nullability under consideration, whereas Java’s overloading guidelines don’t.

The second drawback comes when contemplating that almost all software program growth at Meta — as with anyplace else — entails modifying current code. If most of our code is in Java, we aren’t permitting our builders to totally take pleasure in Kotlin. Because the migration is a protracted course of, anticipating each engineer to transform a file to Kotlin earlier than they contact it’s exhausting and inefficient.

How we’re migrating to Kotlin

We thought-about these two choices and determined our aim can be to transform nearly all our code into Kotlin. After a sluggish begin, the place we needed to repair a number of blockers, we had been capable of start changing loads of code at bulk. In the present day, our Android apps for Fb, Messenger, and Instagram every have greater than 1 million strains of Kotlin code, and the speed of conversion is growing. In complete, our Android codebase has greater than 10 tens of millions strains of Kotlin code.

Unblocking

As quickly as we began making an attempt to make use of Kotlin in our current apps we hit some points. For instance, we wanted to replace Redex to assist bytecode patterns that Java didn’t generate. As well as, some inside libraries we use rely on reworking bytecode throughout compilation to realize higher efficiency. This code didn’t work when run as a part of a Kotlin compilation.

We constructed workarounds for our instruments to unravel these points. Should you migrate your code to Kotlin and have a bunch of in-house optimizations, you must count on related issues. Nonetheless, we count on most individuals is not going to face such points.

We additionally recognized numerous gaps with current tooling. For instance, Kotlin syntax highlighting in our code evaluate or wiki was missing. We up to date Pygments, the library we’re utilizing, to carry the expertise to par with Java. We up to date a few of our inside code-modding instruments to have the ability to deal with Kotlin. We additionally constructed Ktfmt, a deterministic Kotlin formatter primarily based on the code and philosophy of google-java-format.

Accelerating the migration

With our instruments prepared, we might now convert any a part of our code to Kotlin. However every migration required a bunch of boilerplate work that needed to be completed manually. J2K is a normal device and, as such, avoids understanding the code it’s changing. This creates many circumstances that require handbook work.

One common instance is the utilization of JUnit testing guidelines, that are generally utilized in exams.

For instance, you could wish to confirm the right exceptions are thrown utilizing the ExpectedException rule:

@Rule public ExpectedException expectedException = ExpectedException.none();

When J2K converts this code to Kotlin, we get: 

@Rule var expectedException = ExpectedException.none()

This code appears equal at first to the unique Java, however as a consequence of Kotlin’s use website annotations, it’s truly equal to:

@Rule personal ExpectedException expectedException = ExpectedException.none();

public ExpectedException getExpectedException() 
  return expectedException

Attempting to run this take a look at will fail and return an error: “The @Rule expectedException must be public” since JUnit will see a non-public subject annotated with @Rule. This can be a frequent drawback that has been answered many instances in boards and might be mounted in considered one of two methods: both add `@JvmField` to the sphere or add an annotation use-site to the annotation so it’s `@get:Rule`:

// answer 1: use `get` because the use-site for the annotation
@get:Rule var expectedException = ExpectedException.none()

// answer 2: generate JVM code just for a Java subject with out a getter
@JvmField @Rule var expectedException = ExpectedException.none()

Since J2K doesn’t (and possibly mustn’t) know the intricacies of JUnit, it can not do the proper factor. Even when we thought JUnit had been common sufficient that it’d warrant having J2K find out about it, we’d nonetheless have this identical drawback with many area of interest frameworks.

For instance, loads of Android Java code will use the utility strategies from android.textual content.TextUtils, corresponding to isEmpty to simplify the test of some strings. In Kotlin, nevertheless, we’ve the built-in commonplace library technique String.isNullOrEmpty. This technique is preferable not solely as a result of it’s in the usual library, but additionally as a result of it has a contract that tells the Kotlin compiler that if it returns false, the article being examined can not be null and might be smart-cast to a String.

Java code has many different related helper strategies, and lots of libraries implement these identical primary strategies. All of those needs to be changed with the usual Kotlin strategies to simplify the code and permit the compiler to correctly detect nonnullable sorts.

We’ve discovered many situations of those small fixes. Some are simple to do (corresponding to changing isEmpty), some require analysis to determine the primary time (as within the case of JUnit guidelines), and some are workarounds for precise J2K bugs that can lead to something from a construct error to totally different runtime habits.

To resolve these points, we put J2K in the course of a three-step pipeline:

1. In step one, we take one Java bundle and put together it to be transformed to Kotlin. This step principally works round bugs and does conversions wanted for our inside instruments.
2. The second step is working J2K. We’ve been capable of run Android Studio in a headless mode and invoke J2K, which permits us to run the whole pipeline as a script.
3. Within the final step, we postprocess the brand new Kotlin information. This step incorporates nearly all of our automated refactors and fixes steps corresponding to tagging a JUnit rule as a @JvmField. As a part of this step, we additionally apply our autocorrecting linters and apply numerous Android Studio solutions in headless mode.

These automations don’t resolve all the issues, however we’re capable of prioritize the commonest ones. We run our conversion script (aptly named Kotlinator) on modules, prioritizing lively and easier modules first. We then observe the ensuing commit: Does it compile? Does it go our steady integration easily? If it does, we commit it. And if not, we have a look at the problems and devise new computerized refactors to repair them. For points that don’t appear systematic or new, we merely repair them manually and commit the change.

For the Java refactors, we use JavaASTParser, which permits us to resolve some sorts, together with different inside instruments.

For the Kotlin facet, we don’t but have answer that may resolve sorts, so we decide to make use of the Kotlin compiler APIs. Loading a Kotlin code into its PSI AST is easy and, in observe, offers us all the ability we have to repeatedly enhance Kotlinator.

Since we began this course of, we’ve discovered a bit whereas utilizing the Kotlin compiler APIs, so we’re additionally releasing a restricted set of a few of the automated refactorings within the hope that it’s going to assist extra builders use the Kotlin compiler parser to their benefit.

Here’s a fast instance of utilizing a template-matching utility we constructed to deal with the Android TextUtils.isEmpty case talked about above: 

val ktFile = load(path)
// make sure that the right class is imported
if (ktFile.imports.none  
        it.importedReference?.textual content == "android.textual content.TextUtils"
    ) 
    return

val newContent = ktFile.replaceAll(
    matcher = template 
        val a by match 
        "TextUtils.isEmpty($a)"
    ,
    replaceWith = 
        val a by it.variables
        "$a.isNullOrEmpty()"
    )
write(path, newContent)

If in case you have an adversarial thoughts, you possibly can in all probability see a bunch of the way to interrupt this refactor. In observe, we discover that they don’t present up in our code, and that that is sufficient for us to maneuver ahead.

What we’ve discovered from our Kotlin migration

With our tooling enhancements, we had been already capable of convert a large chunk of our code into Kotlin. We have already got greater than 10 million strains of Kotlin code in our codebase, and nearly all of Android builders at Meta at the moment are writing Kotlin code.  

This scale has led us to some conclusions:

Lowered code size

We anticipated Kotlin code to be shorter going into this migration. Some information had been certainly reduce in half (and much more), particularly when the Java code needed to null-check many fields, or when easy repetitive loops could possibly be changed with commonplace Kotlin strategies that settle for a lambda, corresponding to “first,” “single,” “any,” “etc.”

Nonetheless, loads of our code is solely about passing values round. For instance, a Litho class, which defines UI and its styling, stays about the identical size no matter whether or not it’s in Java or Kotlin.

On common, we’ve seen a discount of 11 % within the variety of strains of code from this migration. We’ve seen a lot increased numbers quoted on-line, however we suspect these numbers are derived from particular examples.

We’re nonetheless blissful about this quantity, because the strains eliminated are often boilerplate code, which is much less implicit than its shorter Kotlin counterpart.

Sustaining execution pace

Since Kotlin compiles to the identical JVM bytecode, we didn’t count on to see any execution pace efficiency regressions from this migration.

To confirm this, we ran a number of A/B exams evaluating a Java implementation with a Kotlin implementation, utilizing Kotlin options corresponding to lambdas, nullability, and extra. We discovered that Kotlin matched the efficiency of Java, as we anticipated.

Construct measurement will not be a problem

The Kotlin commonplace library is fairly small, and since all of our releases use Proguard and Redex, solely a few of it even makes it right into a launch APK. Due to this fact, measurement hasn’t proved to be an issue besides in a state of affairs the place a number of KBs of additional code matter. In these circumstances, we discovered that by avoiding Kotlin’s commonplace library and utilizing the already out there Java strategies, the issue might be solved. For instance, utilizing CharSequence.cut up from kotlin.textual content would add a number of lessons and constants compared with utilizing Java’s String.cut up.

Addressing longer construct instances

We anticipated construct instances can be longer with Kotlin because it’s a comparatively new language, in contrast with Java. We guessed proper, and our builders observed that construct instances elevated as we used extra Kotlin in our codebase.

Whereas the Kotlin compiler retains enhancing, we checked out methods we will enhance construct instances on our finish. One in every of them is source-only ABI assist in our Buck construct system, which may generate ABI jars for dependencies within the construct graph with out truly compiling them. That is already supported for Java, and we’re engaged on a Kotlin model, which we imagine will flatten the construct graph and vastly enhance incremental construct speeds.

The opposite space we investigated is annotation processing, a identified ache level for construct pace. Kotlin helps annotation processors utilizing KAPT, which is presently in upkeep mode. KAPT works by producing a Java code stub for the prevailing Java annotation processor code to run. It’s good because it lets your current code work with out modifications, but it surely’s sluggish as a result of era of the Java stub.

The answer is to make use of KSP, the brand new, really helpful approach to deal with annotation processing. We added assist for it in Buck and are engaged on porting our current processors to KSP utilizing an adapter we developed. This does reduce the price of working annotation processors, however provided that no KAPT-based processors stay. The draw back is that this requires loads of work to replace all of the annotation processors. We discovered the interop library by the Room builders as one other choice to reuse current code, however there’s nonetheless obligatory migration work wanted for every processor.

What’s subsequent for Kotlin at Meta?

Our migration to Kotlin remains to be ongoing and accelerating. We’ve been capable of permit any Android developer at Meta who needs to make use of Kotlin to take action and have equipped them with instruments to simply migrate current code to Kotlin.

Kotlin nonetheless lacks a few of the instruments and optimizations that we’ve grown used to from working with Java. However we’re working to shut these gaps. As we make progress and these instruments and libraries mature, we may even work to launch them again to the neighborhood.