AndroidToday

Kotlin K2 and Standalone Source Generator

Posted by Ting-Yuan Huang – Software Engineer, and Jiaxiang Chen – Software Engineer

The Kotlin Symbol Processing (KSP) tool provides a high-level API for doing meta-programming in Kotlin. Many tools have been built on KSP, enabling Kotlin code to be generated at compile time. For example, Jetpack Room uses KSP to generate code for accessing the database, based on an interface provided by the developer, like:

@Dao
interface UserDao {
    @Query("SELECT * FROM user")
    fun getAll(): List<User>
}

KSP provides the API to the Kotlin code so that Room in this case can generate the actual implementation of that interface. While KSP has become a core foundation for meta-programing in Kotlin, its current implementation has some gaps which we are aiming to resolve with a new KSP2 architecture. This blog details those architectural changes and the impact for plugins built on KSP.

In addition, KSP2 has preview support for:

    • The new Kotlin compiler (code-named K2)
    • A new standalone source generator that provides more flexibility and features than the current Kotlin compiler plugin

After getting feedback on the new architecture and continuing to address gaps we will work towards releasing KSP 2.0 where these changes will be the default.

Enabling the KSP2 Preview

The new preview changes can be enabled in KSP 1.0.14 or newer using a flag in gradle.properties:

Note: You might need to enlarge the heap size of the Gradle daemon now that KSP and processors run in the Gradle daemon instead of the Kotlin compiler’s daemon (which has larger default heap size), e.g. org.gradle.jvmargs=-Xmx4096M -XX:MaxMetaspaceSize=1024m

KSP2 and K2

Internally KSP2 uses the Beta Kotlin K2 compiler (which will be the default compiler in Kotlin 2.0). You can use KSP2 before switching your Kotlin compiler to K2 (via the languageVersion setting) but if you want to use K2 for compiling your code, check out: Try the K2 compiler in your Android projects.

Standalone Source Generator

KSP1 is implemented as a Kotlin 1.x compiler plugin. Running KSP requires running the compiler and specifying KSP and its plugin options. In Gradle, KSP’s tasks are customized compilation tasks, which dispatch real work to KotlinCompileDaemon by default. This makes debugging and testing somewhat difficult, because KotlinCompileDaemon runs in its own process, outside of Gradle.

In KSP2, the implementation can be thought of as a library with a main entry point. Build systems and tools can call KSP with this entry point, without setting up the compiler. This makes it very easy to call KSP programmatically and is very useful especially for debugging and testing. With KSP2 you can set breakpoints in KSP processors without having to perform any other / irregular setup tasks to enable debugging.

Everything becomes much easier because KSP2 now controls its lifecycle and can be called as a standalone program or programmatically, like:

val kspConfig = KSPJvmConfig.Builder().apply {
  // All configurations happen here.
}.build()
val exitCode = KotlinSymbolProcessing(kspConfig, listOfProcessors, kspLoggerImpl).execute()

KSP2 API Behavior Changes

With the new implementation, it is also a great opportunity to introduce some refinements in the API behavior so that developers building on KSP will be more productive, have better debuggability and error recovery. For example, when resolving Map<String, NonExistentType>, KSP1 simply returns an error type. In KSP2, Map<String, ErrorType> will be returned instead. Here is a list of the current API behavior changes we plan on making in KSP2:

    1. Resolve implicit type from function call: val error = mutableMapOf<String, NonExistType>()

      KSP1: The whole type will be an error type due to failed type resolution

      KSP2: It will successfully resolve the container type, and for the non-existent type in the type argument, it will correctly report errors on the specific type argument.

    2. Unbounded type parameter

      KSP1: No bounds

      KSP2: An upper bound of Any? is always inserted for consistency

    3. Resolving references to type aliases in function types and annotations

      KSP1: Expanded to the underlying, non-alias type

      KSP2: Not expanded, like uses in other places.

    4. Fully qualified names

      KSP1: Constructors have FQN if the constructor is from source, but not if the constructor is from a library.

      KSP2: Constructors do not have FQN

    5. Type arguments of inner types

      KSP1: Inner types has arguments from outer types

      KSP2: Inner types has no arguments from outer types

    6. Type arguments of star projections

      KSP1: Star projections have type arguments that are expanded to the effective variances according to the declaration sites.

      KSP2: No expansion. Star projections have nulls in their type arguments.

    7. Variance of Java Array

      KSP1: Java Array has a invariant upper bound

      KSP2: Java Array has a covariant upper bound

    8. Enum entries

      KSP1: An enum entry has its own subtype with a supertype of the enum class (incorrect behavior from language point of view)

      KSP2: An enum entry’s type is the type of the enclosing enum class

    9. Multi-override scenario

      For example

      interface GrandBaseInterface1 {
          fun foo(): Unit
      }
      
      interface GrandBaseInterface2 {
          fun foo(): Unit
      }
      
      interface BaseInterface1 : GrandBaseInterface1 {
      }
      
      interface BaseInterface2 : GrandBaseInterface2 {
      }
      
      class OverrideOrder1 : BaseInterface1, GrandBaseInterface2 {
          override fun foo() = TODO()
      }
      class OverrideOrder2 : BaseInterface2, GrandBaseInterface1 {
          override fun foo() = TODO()
      }
      

      KSP1:

      Find overridden symbols in BFS order, first super type found on direct super type list that contains overridden symbol is returned
      For the example, KSP will say OverrideOrder1.foo() overrides GrandBaseInterface2.foo() and OverrideOrder2.foo() overrides GrandBaseInterface1.foo()

      KSP2:

      DFS order, first super type found overridden symbols (with recursive super type look up) in direct super type list is returned.

      For the example, KSP will say OverrideOrder1.foo() overrides GrandBaseInterface1.foo() and OverrideOrder2.foo() overrides GrandBaseInterface2.foo()

    10. Java modifier

      KSP1: Transient/volatile fields are final by default

      KSP2: Transient/volatile fields are open by default

    11. Type annotations

      KSP1: Type annotations on a type argument is only reflected on the type argument symbol

      KSP2: Type annotations on a type argument now present in the resolved type as well

    12. vararg parameters

      KSP1: Considered an Array type

      KSP2: Not considered an Array type

    13. Synthesized members of Enums

      KSP1: values and valueOf are missing if the enum is defined in Kotlin sources

      KSP2: values and valueOf are always present

    14. Synthesized members of data classes

      KSP1: componentN and copy are missing if the data class is defined in Kotlin sources

      KSP2: componentN and copy are always present

New Multiplatform Processing Scheme

When it comes to the processing scheme, i.e. what sources are processed when, the principle of KSP is to be consistent with the build’s existing compilation scheme. In other words, what the compiler sees is what processors see, plus the source code that is generated by processors.

What processors see Kotlin compiler see
ClassA.kt, UtilB.kt, InterfaceC.kt … ClassA.kt, UtilB.kt, InterfaceC.kt … + GeneratedFromA.kt, …

In KSP1’s current compilation scheme, common / shared source sets are processed and compiled multiple times, with each target. For example, commonMain is processed and compiled 3 times in the following project layout. Being able to process all the sources from dependencies is convenient with one exception: Processors don’t see the sources generated from commonMain when processing jvmMain and jsMain. Everything must be re-processed and that can be inefficient.

Flow diagram illustrating sources generated from jvmMain and jsMain processing to commonMain

tasks

inputs

outputs

kspKotlinCommonMainMetadata

commonMain

generatedCommon

kspKotlinJvm

commonMain, jvmMain

generatedCommonJvm

kspKotlinJs

commonMain, jsMain

generatedCommonJs

compileKotlinCommonMainMetadata

commonaMain, generatedCommon

common.klib

compileKotlinJvm

commonMain, jvmMain, generatedCommonJvm

app.jar

compileKotlinJs

commonMain, jsMain, generatedCommonJs

main.js

In KSP2, we plan to add an experimental mode that tries to align to how source sets are compiled in K2 better. All sources can be processed only once with the available new processing scheme:

tasks

inputs

outputs

Resolvable but not available in 

getAllFiles / 

getSymbolsWithAnnotation

kspKotlinCommonMainMetadata

commonMain

generatedCommon

kspKotlinJvm

jvmMain

generatedJvm

commonMain, generatedCommon

kspKotlinJs

jsMain

generatedJs

commonaMain, generatedCommon

compileKotlinCommonMainMetadata

commonaMain, generatedCommon

common.klib

compileKotlinJvm

commonMain, jvmMain, generatedCommon, generatedJvm

app.jar

compileKotlinJs

commonMain, jsMain, generatedCommon, generatedJs

main.js

Please note that Kotlin 2.0 is still in beta and the compilation model is subject to change. Please let us know how this works for you and give us feedback.

KSP2 Preview Feedback

KSP2 is in preview but there is still more work to be done before a stable release. We hope these new features will ultimately help you be more productive when using KSP! Please provide us with your feedback so we can make these improvements awesome as they progress towards being stable.

Source link

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button