How can I include source{d} CE as a library in my project?


I’m in the process of putting together an MVP for a side project I’m working on which is to analyze repositories and identify how they can be broken up into open source and closed source segments. To do this, I’d like to incorporate the source{d} CE platform. Ideally, for the MVP I’d like to:

  1. Pull in one/more target repositories into the application
  2. Parse full repo(s) into UASTs
  3. Query the UASTs to identify linkages between parts of the code (e.g. function declarations and calls)

This is basically to begin exploring/validating the hypothesis that it is possible to utilize MLoC to pull apart existing codebases and programmatically analyze them to identify which blocks are best served as open source and which are best kept as closed source.

I’ve been playing around with the CE, mostly through the UI, but I’d like to include the source{d} resources as a library in my own application. Any help/guidance would be greatly appreciated!


P.S. As I play around more with these tools, I’d love to being contributing to the codebase as well - I love that you use Go!


(this answer contains some parts of this conversation @ sourced-ce/issues/135)

source{d} CE is not meant as a library, […] you might want to consider using the individual components directly […] you might even find that the only component you need is bblfshd.

You could run one bblfsh container:

$ docker run --name bblfshd --privileged -p 9432:9432 bblfsh/bblfshd:latest-drivers

And then you could connect to bblfsh using one of the available bblfsh clients directly from your app.

E.g. if you’re using Go, you could use the go client in your app to parse files with the bblfsh container that you already ran. For this purpose, you could follow this example.