Testing on Jet

[safaci2000]

This is mostly a follow up from some of the back and forth on #387 . Just pulling this into a conversation that'd make things easier to follow.

Let me break this down into a few points, I'm ordering these in what I see as the easter to more complex to address.

1. MySQL client dependency

This might be an issue of a feature that is being deprecated as well as the MySQL client.

A. Currently there's a version of mysql pulled in via circleci.
B. There's a version pulled via docker-compose (mysql:8.0.27) which I think is mostly my issue. (I'm on an M2 chipset and there's no valid manifest for my platform)

Right now, the user/dev needs mysql client to be installed which is not great. There's also an issue with the config: --default-authentication-plugin=mysql_native_password, the mysql_native_password is being deprecated.

[Plugin mysql_native_password reported: ”mysql_native_password’ is deprecated and will be removed in a future release. Please use caching_sha2_password instead’](https://forum.cleavr.io/t/warning-server-plugin-mysql-native-password-reported-mysql-native-password-is-deprecated-and-will-be-removed-in-a-future-release-please-use-caching-sha2-password-instead/1251)

My issue was that my client was too new for the version of MySQL in docker and wasn't able to authenticate. Even if it's slower, I would suggest using the mysql client. The two main ones I ran across of are

• https://github.com/ziutek/mymysql (Pure Go)
• https://github.com/Go-SQL-Driver/MySQL/ (C-lib dependency).

I'd recommend the pure go one to avoid local issues again. I think this only affects running tests/init/ which relies on mysql client explicitly.

2. Test Data Loading (Minor)

It would be nice to use something like dbmate, goose to load the test data, particularly as they both are able to be imported as a library and invoked. I think cockroach is the only one not supported by these tools. This is mostly a convenience since rather than writing all the boiler plate to load data across N DBs you can rely on the migration tools. Or maybe use Jet itself and pre-generated code to load the data? (Though I suppose if jet has a bug and it's loading the data that may hide things away)

example: https://github.com/amacneil/dbmate?tab=readme-ov-file#library

3. testcontainers

there's 2 projects that I'm aware of, but they essentially adhere to the same concept.

• https://testcontainers.com/
• https://github.com/ory/dockertest

I've had better experience with testcontainers but either works. The app itself will start a docker container. You can then load the data, tear down etc as needed. Since all the logic, and versions etc is based on go code, you don't need to worry about making sure the docker versions between circleCI and docker-compose match, and the setup instructions become much easier.

go test -v ./... (assuming test data is present)

You can write it where each test has its own docker container it starts for each DB, loads data, and tears down after the test is done OR a set of shared containers which is similar to how the docker-compose stack works.

---

I think address 1 would be handy, the other issue I have and i'm not sure if it's related or not but I can't get all the tests to pass locally which makes contributing a bit harder to do. I'm not sure if it's the difference between mysql 8.0.27 and 8.0 (forced to change to be able to get a docker image running) or something else.

TestCast

  	Date1:    s"2011-02-02 00:00:00 +0000 UTC",
- 	Time:     s"0000-01-01 00:00:00 +0000 UTC",
+ 	Time:     s"0000-01-01 14:06:10 +0000 UTC",

TestModelColumnComment
TestSQLBuilderColumnComment

Both of these, the generated code doesn't line up to the expectations.

I think this might be a legit bug or OS X being dumb, but the output of:

make jet-gen-mysql

gives me:

//
// Code generated by go-jet DO NOT EDIT.
//
// WARNING: Changes to this file may cause incorrect behavior
// and will be lost if the code is regenerated
//

package model

type Link struct {
	ID          int32   `sql:"primary_key"` // this is link id
	URL         string  // link url
	Name        string  // Unicode characters comment ₲鬼佬℧⇄↻
	Description *string // '"\\%\_
}

which naturally fails the tests since all the unicode chars are broken.

So, if you're open to a PR to address any of these, I can see about addressing any of these, but wanted to get your feedback before putting in the work.

[go-jet]

1. MySQL client dependency

I'm fine with moving MySQL initialization to golang code, as long as it doesn't take much longer than from cli. From my memory when the db initialization file was read and just passed to Exec method it took much longer than when the mysql executable was used.

We are already using https://github.com/Go-SQL-Driver/MySQL/ to run tests.

2. Test Data Loading (Minor)

Libraries like dbmate or goose do not do anything special they just read a file and then call Exec method. We already have that in init.go, and we don't need full migration support; we just need database initialization. So I wouldn't introduce new libraries unless there is some real need for them.

3. testcontainers

I'm fine with testcontainers as long as they aren't significantly slower than the current approach.

Regarding the bugs I took a look at TestCast and it seems it is a driver issue(needs more investigations). I couldn't reproduce TestModelColumnComment, TestSQLBuilderColumnComment so it is probably something specific to OSX.

[safaci2000]

Okay, I'll create some issues related to that and see if I can find some time to address that.

so 1. Testcontainers and 2. Explore using Go-SQL Driver to init the data.

One question, do you know if the driver you linked has a dependency on any C runtime? mymysql seems to explicitly call that out which is why I pointed it out, but if not then that's great. Otherwise the benefit of doing it in go is kind of a moot point if you still need to apt-get install something.

[go-jet]

One question, do you know if the driver you linked has a dependency on any C runtime?

No it is a pure Go.

[safaci2000]

I wanted to revisit this, I have a branch that's mostly working using testcontainers which makes the developer experience easier and simpler and testing. It also makes it easier to get GH action to work with testcontainers rather than circle CI. The down side is it's a bit slower for running single test

So on running a test, it'll spin up a docker container, seed the data and drop into the test. So it makes the whole setup process much easier but it's terribly slow for cockroach. (mysql, mariadb, postgres take about 7-12seconds to load the data and run the Jet mode generation, cockroach takes about 2-3 minutes which I would find too long for a developer experience). Hence my question on here: #401 and the issue on cockroachdb

It also removes the need to use the local mysql client which makes it difficult to work with as installs vary and some clients just don't work with V8 anymore, particularly with the mysql_native_password which has been deprecated for a while and newer clients aren't supporting that password mechanism.

Are the benefits worth the drawbacks and should I keep pursuing this and do you prefer the current flow @go-jet. This of course is dependent on a faster way to load the cockroach DB data.

[go-jet]

Is it possible for testcontainers to retain test data between two test runs?

[safaci2000]

yes, the way I have it setup currently...

TestMain, spins up the container and then uses the same DB instance to run through the tests. Basically the biggest delay is bringing up the test container and seeding the data, after that it runs though all the test for {postgres, mysql, ...etc} and tears down once it's done.

That being said if you're running a single test it's still the same process bring up DB, seed it, and tear down.

tests/init uses the same code for the most part, it just assumes the data is loaded and only runs the generate code pieces.

This is a WIP still but here's the current pipeline: https://github.com/safaci2000/jet/actions/runs/11366172559

Initialize Test data loads all the data, generates the code models then all the various tests run in parallel to validate behavior for all the databases, I have some of the tests disabled there, but the 2nd level would be all the various database supported.

[safaci2000]

The other key benefit I forgot to mention is that you now only have a single location that defines the version of the various DB. You no longer need a docker-compose file, and. you also are not setting up services in circleci or GH actions. It relies on the code to do the right thing and bring up the appropriate version. If you want to change the behavior, you simple change 1 constant in a single location and it gets reflected across the board.

[go-jet]

yes, the way I have it setup currently...

TestMain, spins up the container and then uses the same DB instance to run through the tests. Basically the biggest delay is bringing up the test container and seeding the data, after that it runs though all the test for {postgres, mysql, ...etc} and tears down once it's done.

No, I meant if run locally. Can the docker container retain test data? So it doesn't have to seed the database on each test run.

[safaci2000]

I mean, theoretically yes. It's essentially just docker invoked from code. You could probably expose a volume and mount it based on some condition that can occur. It's just fields weird to have a conditional branch that adds state to tests.

[go-jet]

It seems that the drawbacks outweigh the benefits. Even if we avoid reseeding test data, just starting a docker takes a couple of seconds, and that could prove annoying during development. I think it is better to retain the current flow.