Georg Koppen:

Nicolas Vigier:

...

Hello,

A few days ago on #tor-dev:

01:20 < mikeperry> arthuredelstein,boklm: I'm going into mozilla next wednesday, and I suspect they're going to want to make progress on getting our branch running in their testing infrastructure.. a thought occurrs: what if we were to make an auto-rebasing script that periodically tries to rebase our patches on to master from https://github.com/mozilla/gecko-dev? then we'd not only have early warning if they break our new tests, but also when our patches begin 01:21 < mikeperry> it may mean more work in terms of constantly fixing broken patches though..

I think I can do something to try to automatically rebase our patches on gecko-dev master, build and run the tests every night, then send an email if the rebase failed, the build failed, or a test that was successful the previous day failed.

Sometimes the auto-rebasing will fail. Maybe we can have a 'nightly' or 'master' branch where we push manually rebased patches when auto-rebasing failed ?

I don't know how often the auto-rebasing will fail, and how much work it will be to manually fix it though.

Does it look like a good idea ?

I think before we start with auto-rebasing at all someone should rebase our patches to mozilla-central once as probably a lot of them are failing if we would start with the auto-rebasing right away which would lead to quite some noise we could avoid. Additionally, there are probably patches that need to get rewritten to a great deal (the image cache part comes to mind here).

This is probably already involving quite some work. Then we should come up with a plan on what to do with broken patches. Do we fix them ad hoc or just once every week or...? Who is going to do that? What means "fixed" here at all? Just a successful manual rebasing or does the code need to be compiling as well at least? If so on which platforms? Who is organizing that?

Well what I said immediately after those two lines that Boklm pasted was that we would store conflicts (as full versions of failed patches) in a directory in the repo. In this way, we at least get some advanced notice of conflicts, even if we don't have the resources to keep them up to date continuously.

And most importantly how do we make sure that the functionality of the patches is still the same? I very much fear that subtle rebase problems are carried around (see the potential DNS leakage that occurred during the last rebase) making us think everything is fine where in reality this is not the case.

Well, this we should keep track of via unit tests as best we can.

But you're right, there is a danger that a mis-application could happen. We probably should not treat auto-rebased patches that claim to succeed as the actual final versions. We should instead reapply them fresh from the official versions in our repo.

Manually updated patches we can perhaps mark differently in the repo.

Thus, I am currently not convinced this is a good idea at all. Definitely not until we sort out how this should work in practice.

The benefit is the ability to make more active use of Mozilla's testing infrastructure, and especially the early notice property. Even if many patches end up discarded, we'll at least know that we have a lot of work ahead of us for the next rebase cycle, and that we should plan for that earlier.

We'll also get early notice if a feature change causes a subtle compatibility issue with our patches due to unit test failures, which may be even more valuable.

Mike Perry:

Georg Koppen: Well what I said immediately after those two lines that Boklm pasted was that we would store conflicts (as full versions of failed patches) in a directory in the repo. In this way, we at least get some advanced notice of conflicts, even if we don't have the resources to keep them up to date continuously.

...

And most importantly how do we make sure that the functionality of the patches is still the same? I very much fear that subtle rebase problems are carried around (see the potential DNS leakage that occurred during the last rebase) making us think everything is fine where in reality this is not the case.

Well, this we should keep track of via unit tests as best we can.

Yes, although that is currently hard as we are breaking a bunch of unit tests with our patches (hence there is a lot of noise) and we don't have good coverage of our own features (e.g. we can't test whether the DNS patch is properly applied).

But you're right, there is a danger that a mis-application could happen. We probably should not treat auto-rebased patches that claim to succeed as the actual final versions. We should instead reapply them fresh from the official versions in our repo.

Sounds good to me.

Manually updated patches we can perhaps mark differently in the repo.

Yes.

...

Thus, I am currently not convinced this is a good idea at all. Definitely not until we sort out how this should work in practice.

The benefit is the ability to make more active use of Mozilla's testing infrastructure, and especially the early notice property. Even if many patches end up discarded, we'll at least know that we have a lot of work ahead of us for the next rebase cycle, and that we should plan for that earlier.

We'll also get early notice if a feature change causes a subtle compatibility issue with our patches due to unit test failures, which may be even more valuable.

Those are good points and I agree that this is really worth the setup of an auto-rebase script (+ a little overhead managing the results). I was just afraid that we might want to keep up with rebasing broken patches against mozilla-central all the time and that this effort might be wasting our resources. Gald this was a misunderstanding.

Georg