Make Localization synchronous

[jdesrosiers]

There's one more thing we need in order to generate error messages synchronously. Localization.forLocale is asynchronous because it reads the ftl file with the message templates. We need a way to do this synchronously. Relying on files is problematic for other reasons as well. Bundling is the thing that comes to mind which we'll need for the language server work.

We need to somehow move away from depending on files. We can put the translations in a string that can be imported, but then we loose to ability to use editor tooling for ftl files. It's probably something we'll just need to accept.

[Ahmedmhmud]

I reviewed the localization flow and it can by done by adding a tiny script that turns them into a generated JavaScript translations map will probably look like this

export const translations = {
  "en-US": `...full FTL text...`,
  "fr-FR": `...full FTL text...`
};
// "fr-FR" is an example for future potential additions

then make localization read from that in-memory map by importing the generated module instead of reading files at runtime. Then I will make Localization.forLocale synchronous because translation data is already loaded, keep the same errors for unsupported locales and missing messages, update tests to stop creating temporary translation files, and run the generator before tests so everything stays in sync.

[jdesrosiers]

I have a strict no-build-step policy for libraries. (That's why we use JSDoc for types rather than use TypeScript directly.) So, having a script that builds the translations object isn't an option.

It occurred to me that we could read all the files into translations using a top-level await. That would allow us to make Localization.forLocale synchronous, but we still have the bundling problem with any solution that relies on non-js files.

[Ahmedmhmud]

There is a simple solution, and I think it is the most appropriate one. We can keep the .ftl files for editing, and also create a JS module like src/translations.js from them for runtime use exactly like the one we wanted to generate by the script. When we update the .ftl files, we update the JS module too. This gives us the .ftl files for editing and the JS module for runtime, so even though it is a bit redundant, I think it fits our case well.

[jdesrosiers]

When we update the .ftl files, we update the JS module too.

Do you mean manually make the same change in both places? That's not a great solution. That's still a build step, it's just a manual one.

I'm pretty sure the only options here are to either break the no-build-step rule or give up being able to use .ftl files.

[srivastava-diya]

I'm pretty sure the only options here are to either break the no-build-step rule or give up being able to use .ftl files.

Hey @jdesrosiers , I’m curious about the reasoning behind the strict no-build-step policy. What advantages does it provide in a library like this? I’m trying to better understand the trade-offs.

[Ahmedmhmud]

I think we should go with keeping the no-build-step policy. If we did a .ftl to JS copy flow, that would still be a build step and could make publishing more confusing. Since we have already chosen this policy, the most suitable solution is to stop using .ftl files at runtime and keep translations directly in a JS module as the single source of truth. That gives us synchronous localization without adding build tooling.

[jdesrosiers]

I’m curious about the reasoning behind the strict no-build-step policy. What advantages does it provide in a library like this? I’m trying to better understand the trade-offs.

I want the code to be able to run anywhere as is. There are two cases I run into frequently that it enables.

One of them is debugging. When I need to dig into why something isn't working, sometimes I need to follow the code execution into dependencies. If that leads me to generated code, it's way more difficult to figure out what's going on than code crafted to be readable. It also makes it harder to make fixes if bugs are found. It makes everything easier if I can work with the actual source code.

The other case is previews of unpublished work. Sometimes, I want to test changes to a library in the context of another library that uses it before I publish. A handy way to do that is to use github identifiers to install the module from a branch (Example: "@hyperjump/json-schema-errors": "github:hyperjump-io/json-schema-errors#experimental"). However, using this method has some limitations. For some reason, installing from github doesn't support all the lifecycle hooks that npm usually supports, like post-install scripts that would normally be used to do the build steps.

There are other ways to deal with these issues, but they have their own problems and involve more complexity and therefore are more error prone and harder to maintain. So, I've settled on the no-build-step policy and it generally works very well. This is a unique challenge I haven't run into yet.

[jdesrosiers]

So, that said, I think I want to stick no-builds for now and put the translations in a string in a .js file. If it becomes a significant pain point in the future, we can always reevaluate that decision.

@Ahmedmhmud Go ahead with that approach.

[srivastava-diya]

One of them is debugging. When I need to dig into why something isn't working, sometimes I need to follow the code execution into dependencies. If that leads me to generated code, it's way more difficult to figure out what's going on than code crafted to be readable. It also makes it harder to make fixes if bugs are found. It makes everything easier if I can work with the actual source code.

The other case is previews of unpublished work. Sometimes, I want to test changes to a library in the context of another library that uses it before I publish. A handy way to do that is to use github identifier..........

That was very insightful, thanks!