So it looks like this issue is /etc/ssh/ssh_config:

  Host *
    UserKnownHostsFile /home/chronos/user/.ssh/known_hosts

Zdenek already fixed this, but Portage is protecting the old version of the file using config_protect... so how do we ensure the update gets through?

A workaround would be to remove the UserKnownHostsFile entry manually from /etc/ssh/ssh_config

One fix discussed with davidjames is to remove config-protect in chroot entirely. That is currently dangerous as it will clobber not only the good files but also the bad files and can make the chroot even more broken.

Discussed this with vapier@. We just ran into a similar issue where config-protect was causing an openssl update to not take effect, thus both masking issues and further not allowing fixes to take effect. We should reduce the scope of config protect so that it doesn't hide so many problems.

If we want to disable config protect entirely, we'll need to fix any cases where we depend on files being modified manually by scripts, such as in make_chroot.

(No comment was entered for this change.)

Commit: c8acc7f6639b6d04fdea42308fd0959085574088
 Email: zbehan@chromium.org

cros-overlay: redefine CONFIG_PROTECT variables to make a little more sense

For the most part, CONFIG_PROTECT is a hassle and is being explicitly
overriden wherever possible, but not quite enough. This introduces an
opposite approach. First clear all CONFIG_PROTECTS and then explicitly
list the files that we need protected in the chroot target, because they
are initialized in make_chroot and their clobber would break the chroot.

BUG=chromium-os:13987
TEST=below, verifying that various build enviroments get the right settings
1) emerge --info
CONFIG_PROTECT="/etc/hosts /etc/locale.gen /etc/localtime /etc/make.conf
/etc/sudoers /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d
/etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild
/etc/terminfo"
2) emerge-x86-generic --info
CONFIG_PROTECT="/etc/make.globals /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d
/etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild
/etc/terminfo"

Change-Id: Iad0b0cb2e8a9359fb1091cfeeffd0a013e5ec4e7
Reviewed-on: http://gerrit.chromium.org/gerrit/7805
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>

M	chromeos/config/make.conf.common-target
M	profiles/base/make.defaults
M	profiles/default/linux/amd64/10.0/make.defaults
M	profiles/targets/chromeos/make.defaults

I'm a bit uncomfortable with just flat out overriding all conflicts; at least for the ssh case, any reason ssh isn't adding an appropriate CONFIG_PROTECT_MASK ?

Not sure what you mean by conflicts. Anyway, in general, this is a problem that has unfortunately been rolling since as long ago as I remember, and all config updates in the chroot were being simply ignored and piling up (very bad).

The way I see it, in the ideal case, packages that need to be bent to provide a cros-specific config files should be bent to install our desired config files, and we shouldn't need CONFIG_PROTECT at all, as updating those requires running an interactive tool after each world update. There's a little bit of way to go for that though, many config files are still generated and completely incompatible with the stock versions.

I'm trying a step towards that, solving several specific problems at hand that arose as a result of neglecting the config updates issue completely, but I hope we agree about the ultimate goal, if not, we should probably sit and discuss it.

Pardon on the terminology; Conflicts in this sense means anything CONFIG_PROTECT would protect; a new version of the package trying to merge a file that has a different md5 (conflicting) from what's on disk.

CONFIG_PROTECT is unfortunately a bit stupid; the decision of whether or not a file should be protected, or overwritten is purely based on the file lieing withing a CONFIG_PROTECT'd pathway (minus the CONFIG_PROTECT_MASK intersection); thing is, CONFIG_PROTECT for a users system is different from our build envs/roots.  Simplest question, why are we even protecting /etc/ssh/ for example?  Whatever the package provides, most likely we want that- including whatever modifications we've folded into the pkg.  If a new version has a new config file, we don't want protection, it should just be updated.

As such, we should be adding CONFIG_PROTECT_MASK for those pathways.

A second notion here is that portages configuration protection, at least in our usage, likely should be backed down in it's strictness.  If the md5 of the on disk file matches what was previously merged (the old version), then, in our usage, I'd argue it's valid to just replace the file- it wasn't changed after all.

Basically, I'm uncomfortable with the nuclear approach (forced automerge on all); there can be cases where it's undesired.  I'd rather see us eliminate the issue in general for our usage- set an appropriate CONFIG_PROTECT_MASK, from there tweak portage to be less agressive in protection, /then/ look at the etc-update issue.

Via this approach, we should be able to reduce it down to just a couple of files where we actually care- things like portage's default make.globals changing, or baselayout/openrc upgrades, that sort of thing.

If you take a look at the previous commit (mentioned in this issue), that is precisely what I did. Replace default CONFIG_PROTECT with "-*" and then explicit listing of files we care about (For now).

That's also why the update script i posted has two bits, pwn all updates (because we care about keeping the current versions) and automerge the rest.

Trying a reverse approach by keeping original CONFIG_PROTECT and then listing everything else in CONFIG_PROTECT_MASK is an option, but very time consuming, not playing nice with keeping in sync with upstream (we'd have to modify ebuilds to export their _MASKs), and probably not worth the effort, IMO.

yeah, i would lean towards only CONFIG_PROTECT-ing the few things we care about, and letting all the rest get clobbered during the emerge.  the default CONFIG_PROTECT values make sense when Gentoo is your host OS, but not really at all when talking about the SDK chroot.

Commit: 3c1d3d8be6f0f8b3cc34fb762f997b39d4cc9c36
 Email: zbehan@chromium.org

update hooks: add a hook for fixing all pending config updates

BUG=chromium-os:13987
TEST=below
1) export CONFIG_PROTECT="/etc /usr/share"
2) checkout as old as possible revision of cros-overlay, update chroot to it
3) checkout HEAD and update back
4) run the hook, see it update thousands of files correctly

Change-Id: I4213de8ccba6e573541e96591ea0d033a10218a5
Reviewed-on: http://gerrit.chromium.org/gerrit/8938
Tested-by: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>

A	chroot_version_hooks.d/12_clean_old_config_files

Commit: eeaf674db40f5c86bb516d7e5404b5c3006dbe02
 Email: zbehan@chromium.org

update hooks: 3c1d3d8be6f0f8b3cc34fb762f997b39d4cc9c36, take two

add a hook for fixing all pending config updates

BUG=chromium-os:13987
TEST=below
1) export CONFIG_PROTECT="/etc /usr/share"
2) checkout as old as possible revision of cros-overlay, update chroot to it
3) checkout HEAD and update back
4) run the hook, see it update thousands of files correctly

Change-Id: Idabf5475516ed58b76ceee6ffdc817a5f46839ad
Reviewed-on: http://gerrit.chromium.org/gerrit/10022
Reviewed-by: Sameer Nanda <snanda@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>

A	chroot_version_hooks.d/12_clean_old_config_files

(No comment was entered for this change.)

Move to next iteration.

The bulk of the work is done, closing this completely depends on http://gerrit.chromium.org/gerrit/#change,10166 which is complete and only waiting for a dependent change.

Commit: 99fb742e5a903680777640d198dec0d6fbfd03cc
 Email: zbehan@chromium.org

update_chroot: autodiscard all config updates

* This is now safe to do, since the only protected files are exactly
the ones that have been autogenerated during chroot creation and may
not be overwritten.

BUG=chromium-os:13987
TEST=update the chroot

Change-Id: Ica4d8328b21089b23e2b0e8a29530cedddabc299
Reviewed-on: https://gerrit.chromium.org/gerrit/10166
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Reviewed-by: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>

M	update_chroot

All unwanted updates are now automatically discarded, all wanted updates automatically merged. Closing.

(No comment was entered for this change.)

(No comment was entered for this change.)

(No comment was entered for this change.)

(No comment was entered for this change.)

(No comment was entered for this change.)