This is fascinating. It checks a lot of my boxes. I was recently looking at ActionsFlow [0] which is similar but runs on GitHub Actions.
My thoughts -
1. I don't see proper secret storage being handled. You typically don't want your API keys in your code. What would you recommend instead?
2. "OAuth" based secrets. Many integrations require giving access to an App via OAuth, which involves a flow. I think that's being handled internally from the video [1] and from this project [2], but it's not clear. How is that handled?
A common use case I'd automated once is that when a GitHub project gets starred, the developers public information is scrapped and they are then followed on Twitter, if their twitter handle is found. With Trigger.dev, the twitter part isn't clear.
3. Error Handling - What about when some job fails to run? I understand there is a delay mechanism. But what about injecting custom error handling? Sending a message on slack, for example.
4. Dashboards - They look awesome. And I get the impression that each "action" in the code is mapped to individual blocks in the dashboard. I'd love to be able to see a proper graph of the flow.
I love that I can see the json request / response for each. It'll make debugging easier when some API changes or fails.
5. No Code solutions - In the long run, I can easily picture writing the integration I want in plain text, and having Github CoPilot or ChatGPT generate the code for me, and then I can quickly modify it.
6. Incentive for integrations - As with most automation tools, entering the market is challenging as you're lacking integrations. The awesome thing about ActionsFlow [0] was that it was re-using an existing community of GitHub Actions, and therefore you don't start from scratch. Have you thought about reusing workflows from n8n or other projects?
7. Integrating with existing Automations - I think a bit more focus should be made on integrating with IFTTT / Zapier / n8n. I see you provide webhooks, but I think some easy wrappers + documentation would be better. This way, I can try out newer workflows in Trigger, and easily just extend my existing system. And then if Trigger.dev works for me, I can think about migrating away from my existing automation solution.
8. Open Source Longevity - Trigger.dev is MIT licensed. Could you please explain the rationale? How do you plan to combat someone launching a competitor using your code? N8n is deliberately "Soure Code Available" and not "Open source", which I thought was a decent compromise. Will you be following a more Open Core model similar to GitLab (which is also MIT licensed)?
9. De-coupling runners and the dashboard - I'd love to not have the pain of maintaining the dashboard / event listeners, but being able to control the running of the jobs. Similar to a CI or Airflow.
10. Support for other languages - This is something that Dagger CI [3] now allows. Letting you use whatever programming language. With Github Actions, I can just package it as a container. Do you plan to support anything else?
After moving from using HCL to Typescript for my Terraform code, the advantage is so great, that I can't seem myself going back to using a custom language such as Dagger's CUE [4]. Trigger.dev targeting Typescript is already a big win. However, I do have a number of automations in Python.
Overall, I'm super optimistic. Congratulations on the launch.
We support both typescript and python at windmill [1]. But our model is doing low-code for the workflows, and actual scripts/code for the steps themselves so it's quite different than trigger.dev. We handle OAuth, refresh tokens and secrets being managed by the platform. It's quite different from trigger.dev considering they are doing workflows entirely in code which has some pro and cons.
Thanks for such an amazing reply, I'll do my best to answer everything. I am also on the Trigger.dev team btw.
1. Your API key with us can either be passed into the Trigger constructor or you can use our TRIGGER_API_KEY environment variable. For the API integrations we provide, we handle API keys for you and they're added inside the dashboard UI. For everything else: as the workflows and files are just code on your server you can use environment variables (or your preferred alternative) to inject secure values.
2. When a workflow step that requires OAuth is hit, the workflow pauses and prompts you to sign
in. After you've signed in the workflow continues where it left off. You only need to connect each service once per organization. You can sign optionally in multiple times to the same service and switch connections where needed, e.g. multiple separate Slack workspaces.
GitHub star to Twitter follow is a great idea. You could achieve that right now by using our GitHub integration and our fetch call (which auto-retries with exponential back off and logging in the dashboard). We'll add a Twitter integration soon so this is really easy and publish it as an example :)
3. We have detailed error messages attached to the step that failed but don't currently have a way to hook into this like you describe. This is a great idea, I've added it to our task list.
4. Exactly right, each "step" is a block in the UI with clear inputs and outputs. We would really like to have a graph view. That'll be especially useful when you put loops in the code, and of course branching.
5. 100%, making this work well with AI code generation would be great.
6. This is a great idea, we'll investigate. Rather than just mapping 1:1 with an SDK we're trying to make the experience better. For example, with the normal Slack API you can't post to a channel name, you have to use an id. We make it so you can use the name and we deal with the hassle for you.
7. We'll look into this too. Integrations for interoperability could make this easier.
8. We're going to follow the GitLab/PostHog model. Like GitLab, the repository as a whole is MIT licensed but we can add some /ee folders in the future with enterprise features. We feel it gives us the right balance of being open source and gives us some protection from a competitor hijacking the project.
9. We will separate out the dashboard from the runners – this was a compromise so we could ship the first version faster.
10. We can relatively easily ship other languages SDKs and Python is probably where we'd start. Our core backend code will most likely remain in TypeScript.
"4. Exactly right, each "step" is a block in the UI with clear inputs and outputs. We would really like to have a graph view. That'll be especially useful when you put loops in the code, and of course branching."
I can't help but think that you want a state machine here, rather than imperative code that's being analysed to try and get some kind of a graph out of it..
I'm personally looking forward to a cloud, state machine based service that offers the whole long running workflow thing. The closest I've seen so far was people using xstate within temporal.io workflows.. but would be amazing to have really nice observability out of the box, with all the clarity that state machines bring.
My thoughts -
1. I don't see proper secret storage being handled. You typically don't want your API keys in your code. What would you recommend instead?
2. "OAuth" based secrets. Many integrations require giving access to an App via OAuth, which involves a flow. I think that's being handled internally from the video [1] and from this project [2], but it's not clear. How is that handled?
A common use case I'd automated once is that when a GitHub project gets starred, the developers public information is scrapped and they are then followed on Twitter, if their twitter handle is found. With Trigger.dev, the twitter part isn't clear.
3. Error Handling - What about when some job fails to run? I understand there is a delay mechanism. But what about injecting custom error handling? Sending a message on slack, for example.
4. Dashboards - They look awesome. And I get the impression that each "action" in the code is mapped to individual blocks in the dashboard. I'd love to be able to see a proper graph of the flow.
I love that I can see the json request / response for each. It'll make debugging easier when some API changes or fails.
5. No Code solutions - In the long run, I can easily picture writing the integration I want in plain text, and having Github CoPilot or ChatGPT generate the code for me, and then I can quickly modify it.
6. Incentive for integrations - As with most automation tools, entering the market is challenging as you're lacking integrations. The awesome thing about ActionsFlow [0] was that it was re-using an existing community of GitHub Actions, and therefore you don't start from scratch. Have you thought about reusing workflows from n8n or other projects?
7. Integrating with existing Automations - I think a bit more focus should be made on integrating with IFTTT / Zapier / n8n. I see you provide webhooks, but I think some easy wrappers + documentation would be better. This way, I can try out newer workflows in Trigger, and easily just extend my existing system. And then if Trigger.dev works for me, I can think about migrating away from my existing automation solution.
8. Open Source Longevity - Trigger.dev is MIT licensed. Could you please explain the rationale? How do you plan to combat someone launching a competitor using your code? N8n is deliberately "Soure Code Available" and not "Open source", which I thought was a decent compromise. Will you be following a more Open Core model similar to GitLab (which is also MIT licensed)?
9. De-coupling runners and the dashboard - I'd love to not have the pain of maintaining the dashboard / event listeners, but being able to control the running of the jobs. Similar to a CI or Airflow.
10. Support for other languages - This is something that Dagger CI [3] now allows. Letting you use whatever programming language. With Github Actions, I can just package it as a container. Do you plan to support anything else?
After moving from using HCL to Typescript for my Terraform code, the advantage is so great, that I can't seem myself going back to using a custom language such as Dagger's CUE [4]. Trigger.dev targeting Typescript is already a big win. However, I do have a number of automations in Python.
Overall, I'm super optimistic. Congratulations on the launch.
[0] - https://actionsflow.github.io/
[1] - https://www.youtube.com/watch?v=aFlwD0frvnQ
[2] - https://github.com/triggerdotdev/Pizzly
[3] - https://dagger.io/
[4] - https://docs.dagger.io/1215/what-is-cue/