as the backports are growing over time, I wanted to kindly ask whether it would be possible to split them into smaller, more focused chunks.
From our side, maintaining upstream updates against our local changes becomes increasingly difficult, as we carry quite a few project-specific adaptations that would not really make sense to upstream.
What does the Community think about it?
This is of course just a suggestion, and we fully understand if this does not fit your current workflow.
I understand the problem and I am open to improve that situation. As long as it does not slow down the OpenEMS project. FENECON is still the main contributor - and I generally prefer pushing new features upstream ASAP to push the project forward.
it would be great to split the Backports up! I guess this would not slow down, right?
I always Cherry-Picked in the Past - still i often face the Problem that in a increasing ammount of Files it comes to Merge Conflicts which would be easier to resolve if not as much Code would have been changed at once
Thanks again for your consideration!
Is it possible to see the Script or to get it from you?
Not having any own personal stakes in this discussion (and aiming to keep our instance 100% upstream), I do remember, that during the last Hackathon some people from voltfang were stating the same. As far as I understood, they would also prefer to have backports as single commits instead of one big chunk as it would make it easier for them to merge it into their private repo.
I don’t know if making it easier for other people maintaining their private repos, would lead to more contributions from their end.
(In case the discussion gets more complex, discussing it on a hackathon or a general assembly might be a way to proceed.)
The purpose of the script is simply to quickly cherry-pick commits from our internal fork and cleanup the commit message (with separate cleanup-commit-pipe.sh i.e. replacing authors with their github IDs).
I also use a script to write the Pull Request description text for the backport
#!/bin/bash
FILE=`mktemp`
HASHES=`git rev-list develop^..HEAD`
GITHUB_URL="https://github.com/OpenEMS/openems/commit"
get_message() {
local hash="$1"
git log -1 --pretty=%s "$hash"
}
log_message() {
local hash="$1"
MESSAGE=$(get_message "$hash")
echo " * ${MESSAGE} [Commit]($GITHUB_URL/$HASH)" >> $FILE
}
add() {
local name="$1"
local remaining_hashes=""
echo "* ${name}" >> $FILE
for HASH in $HASHES; do
MESSAGE=$(get_message "$HASH")
if [[ "$MESSAGE" == *"[${name}]"* ]]; then
log_message "$HASH"
else
remaining_hashes="${remaining_hashes} ${HASH}"
fi
done
echo "" >> $FILE
HASHES="$remaining_hashes"
}
add "Backend"
add "UI"
add "Edge"
# Remaining entries
echo "* Common" >> $FILE
for HASH in $HASHES; do
log_message "$HASH"
done
code $FILE
This discussion is related to
I understand that it can be complicated to stay synchronized with upstream. I think we should try to improve the tools, scripts, etc. to simplify these processes for everybody.
Jumping in from the Voltfang side, building on sjjh point above, and thanks for sharing the scripts, @stefan.feilmeier, plus the openness to improving this.
One angle I don’t think has come up yet: beyond the merge-conflict pain for fork maintainers, flattening a backport into a single commit also costs the whole community the following things on develop itself:
git blame points every line to a “FEMS Backports” commit instead of the change that introduced it, so you can’t see immediately why a line is the way it is. You have to first infer what part of the backport changed that line.
git bisect can only narrow a regression down to a whole batch (e.g. #3825 = 247 files) instead of the single commit that caused it.
The nice part: the script already commits each change individually (the cherry-pick loop), so they’re all sitting on the backport branch. The only thing collapsing them is the “Squash and merge” at the end. If the PR were landed with “Rebase and merge” (or a merge commit) instead, those commits would land on develop as-is, blame/bisect work again, and it’s ~zero extra effort since nothing about the script or workflow has to change.
Either “Rebase and merge” or “Create a merge commit” fixes this. Both keep the individual commits, so blame and bisect work again. I’d personally lean toward “Rebase and merge” for a linear history that matches the rest of develop, but a merge commit is completely fine too. The only one to avoid is “Squash and merge”.
As a bonus, this also lands each backport in the smallest possible pieces for free - one commit per change - which is the “smaller chunks” idea taken to its natural limit, as long as the commits themselves are reasonably scoped.
