Repoteus: Zynot Tree Vision

[Zach Welch]

Hi all,

Here's my vision for the Zynot trees:

http://wiki.zynot.org/zynot/moin.cgi/ZynotSvn_2fRoadMap

which refers to the recently updated layout:

http://wiki.zynot.org/zynot/moin.cgi/ZynotSvn

It's what a lot of people have been waiting for me to produce, but I
hope that it will give everyone a complete vision of what we need to do.

I really look forward to any final discussions on these matters and
getting our first "official" repositories on a new server. I expect we
will have a new server ready by the end of the week from which we can
provide mirroring and initial downloads. This is our top goal.

I think it will be a short leap from there to get our first set of stage
tarballs and liveCDs ready. In fact, I think we can still readily make
our LWESF goals, but we may be burning disks madly the night before the
show opens.

Cheers,

Zach

[Mark Guertin]

Post by Zach Welch
Hi all,
http://wiki.zynot.org/zynot/moin.cgi/ZynotSvn_2fRoadMap
http://wiki.zynot.org/zynot/moin.cgi/ZynotSvn
It's what a lot of people have been waiting for me to produce, but I
hope that it will give everyone a complete vision of what we need to do.

You should work some of this stuff out with nall as not all of this
functionality exists in porteus yet, and some of it may be substantial to
get things going (i.e. The tree matching the sync, mirror matching sync,
etc).

Post by Zach Welch
I really look forward to any final discussions on these matters and
getting our first "official" repositories on a new server. I expect we
will have a new server ready by the end of the week from which we can
provide mirroring and initial downloads. This is our top goal.
I think it will be a short leap from there to get our first set of stage
tarballs and liveCDs ready. In fact, I think we can still readily make
our LWESF goals, but we may be burning disks madly the night before the
show opens.

LiveCD is a Gentoo thing, which they have been very adamament and vocal
about us not using, at least from PPC land. There will be no PPC live CD
for this time, at least until I have time to write a new one and get it
tested.

I can pretty much for sure say that PPC won't be ready by LWSF, the trees
are in sorry shape and still working away at testing them here, but can't
really continue until these repo's are up and running, but we'll see how it
goes I suppose.

As for stage tarballs this is another discussion we have to bring up,
because I think the way that Gentoo did things in this area was pretty messy
and I think a single stage2 (base) and some binary pkgs would be much more
suitable. This would also cut down installation and distro build times
immensely.

Mark

[jesse]

Post by Mark Guertin
As for stage tarballs this is another discussion we have to bring up,
because I think the way that Gentoo did things in this area was pretty messy
and I think a single stage2 (base) and some binary pkgs would be much more
suitable. This would also cut down installation and distro build times
immensely.

One interesting thought is renaming binary builds to include the use
flags and arch i.e:

evolution-1.2.4_ssl_mozilla_ldap_spell_pda_kerberos-pentium4.tbz

this isn't directly related to the installer but more to the maintenance
of packages across lots of machines. Having the ability to build
descriptive binary packages and to install them when the arch/use flags
match up would be a boon. Instead ppl have to make sure they are using
the exact use flags across multiple machines to leverages binary
packages.

Dunno if its good or what, but I imagine there are only a few ~3-5 ways
most packages will get built.. i used evolution as an example because
its got a huge amount of use flags.

so heres how this _could_ tie into a install process ( and where
something like the p2portage idea _could_ be interesting )

if we extend poteus or porteus2 to have the ability to grab a digest of
packages on the distmirrors that includes binary builds. If the arch/use
flags match up for that _release_ (would have to tie releases to
gcc/glib updates i would think ) then the user could just fetch the
prebuilt.. or chose what prebuilt they want from a list of whats there
or somthing to that effect.

all optional of course, but its more flexible than GRP.

Just somthing the above made me think about.

Post by Mark Guertin
Mark
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

[Mark Guertin]

Post by jesse
One interesting thought is renaming binary builds to include the use
evolution-1.2.4_ssl_mozilla_ldap_spell_pda_kerberos-pentium4.tbz

This was suggested with Gentoo many many times as well.

I for one would like to see a much diminished set of static USE for binary
packages (unless they are custom deployment type stuff), maybe setup in a
profile. This kind of name mangling can get confusing very quickly, and
given the amount of variations on USE that can be present trying to offer
binaries would be next to impossible, 3000 pkgs w/ average of 5 USE per pkg
... that makes for big math given all the possible combinations :(

Not saying it's a bad idea, just saying that I don't think we should put our
foot into that deep of water (yet). I would much prefer to spend our time
targeting ways to do them from a profile, and providing people the tools to
do this for their own custom deployments.

Mark

[Henry Jen]

Post by Mark Guertin

Post by jesse
One interesting thought is renaming binary builds to include the use
evolution-1.2.4_ssl_mozilla_ldap_spell_pda_kerberos-pentium4.tbz

This was suggested with Gentoo many many times as well.
I for one would like to see a much diminished set of static USE for binary
packages (unless they are custom deployment type stuff), maybe setup in a
profile. This kind of name mangling can get confusing very quickly, and
given the amount of variations on USE that can be present trying to offer
binaries would be next to impossible, 3000 pkgs w/ average of 5 USE per pkg
... that makes for big math given all the possible combinations :(

I would like to see binary feeds which will be able to provide
binaries built from a fixed USE flag. There can be many feeds with
different USE setting out there on the internet, and user will be able
to choose a feed match to his USE setting most.

For those user who don't want to spend a lot time to complie their own
packages, a feed can provide them binaries packages. Otherwise, user
can build from source code with ebuild.

Henry
==
known as slowhog on irc.freenode.net
==

[Daniel Armyr]

If we decide to support multiple trees from which one can sync, then the feature would be very useful. While the public servers need only provide packages compiled with a lesser amount of useflags, a large private network could have its own set of precompiled binaries compiled with the useflags the sysop feels like.

My 2 ören

--Daniel Armyr

Post by Mark Guertin
I for one would like to see a much diminished set of static USE for
binary packages (unless they are custom deployment type stuff), maybe
setup in a profile. This kind of name mangling can get confusing very
quickly, and given the amount of variations on USE that can be present
trying to offer binaries would be next to impossible, 3000 pkgs w/
average of 5 USE per pkg... that makes for big math given all the
possible combinations :(
Not saying it's a bad idea, just saying that I don't think we should
put our foot into that deep of water (yet). I would much prefer to
spend our time targeting ways to do them from a profile, and providing
people the tools to do this for their own custom deployments.
Mark
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

[jesse]

Post by Mark Guertin
Not saying it's a bad idea, just saying that I don't think we should put our
foot into that deep of water (yet). I would much prefer to spend our time
targeting ways to do them from a profile, and providing people the tools to
do this for their own custom deployments.
Mark

Im not saying its a good idea :P but its somthing i could use in day to
day opperations. As it is i just sync the make.conf umong all my like
servers .. so that the packagedir thats shared between them is the same,

Post by Mark Guertin
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

[Mark Guertin]

Post by jesse
Im not saying its a good idea :P but its somthing i could use in day to
day opperations. As it is i just sync the make.conf umong all my like
servers .. so that the packagedir thats shared between them is the same,

Right, but this also brings in the larger discussion of p2p ability that a
lot of people are pushing to have. While it's a neat concept it's also a
sysadmin's _nightmare_. I run a fairly large LAN here and I would never in
_any_ way allow this sort of installation (pkgs from from who knows where)
to be installed on any of my computers.

People will argue the point that they are signed, etc... But that doesn't
stop someone from building a trojaned binary and singing it. It just makes
my spider senses tingle thinking about it....

Just more fuel for the fire... ;)

Mark

[jesse]

Post by Mark Guertin

Post by jesse
Im not saying its a good idea :P but its somthing i could use in day to
day opperations. As it is i just sync the make.conf umong all my like
servers .. so that the packagedir thats shared between them is the same,

Right, but this also brings in the larger discussion of p2p ability that a
lot of people are pushing to have. While it's a neat concept it's also a
sysadmin's _nightmare_. I run a fairly large LAN here and I would never in
_any_ way allow this sort of installation (pkgs from from who knows where)
to be installed on any of my computers.
People will argue the point that they are signed, etc... But that doesn't
stop someone from building a trojaned binary and singing it. It just makes
my spider senses tingle thinking about it....
Just more fuel for the fire... ;)

If they are isgned by our devs who are on your keyring .. then you
should be able to trust. Esp if it's got 2 or more valid sigs.

Over any distrib medium a package that had this requirement would be
considered pristine. You would be able to check those sigs agains
keyservers as well as your local copy of the pubkeys.you could trust
that package if all those check were ok.

If random joe-schmo builds a binary and pops it on p2portage signed. It
doesnt give it any more credability in the eyes of porteous. ( not part
of or keysystem ).

Post by Mark Guertin
Mark

[Mark Guertin]

Post by jesse
If they are isgned by our devs who are on your keyring .. then you
should be able to trust. Esp if it's got 2 or more valid sigs.
Over any distrib medium a package that had this requirement would be
considered pristine. You would be able to check those sigs agains
keyservers as well as your local copy of the pubkeys.you could trust
that package if all those check were ok.
If random joe-schmo builds a binary and pops it on p2portage signed. It
doesnt give it any more credability in the eyes of porteous. ( not part
of or keysystem ).

That given I would still never allow this on my system, signed or not
signed. Also this would divert a lot of dev attention from working on the
distro proper.. Instead they would be building (and yielding TONS of emails
from users if they signed the builds as good and they are not). I know this
from experience doing this stuff with Gentoo.

IMO this is something we should still avoid for now.

If you haven't noticed I have problems with the whole concept of p2p
distribution of anything for an operating system, and I am not alone on this
stuff.

If p2p ever gets implemented I for one will want HUGE guarantees that the
option not only be able to be disabled, but that there also be a version of
the pkgmgr that this option literally does not even exist in. One user
turning on something like this unchecked can have massive repercussions on a
large LAN. It goes against the grain in corporate environment in a very big
way here.

Potentially this could make M$ email virii look tame from a security
standpoint...if one signed pkg gets compromised and distributed as trusted
(signed) into a p2p system. With p2p you have zero control after it has
gotten into nodes, that compromised build could live forever, revoking or
no. At least on (semi) controlled mirrors we can properly revoke it if the
need should ever arise.

Mark

[jesse]

Post by Mark Guertin

Post by jesse
If they are isgned by our devs who are on your keyring .. then you
should be able to trust. Esp if it's got 2 or more valid sigs.
Over any distrib medium a package that had this requirement would be
considered pristine. You would be able to check those sigs agains
keyservers as well as your local copy of the pubkeys.you could trust
that package if all those check were ok.
If random joe-schmo builds a binary and pops it on p2portage signed. It
doesnt give it any more credability in the eyes of porteous. ( not part
of or keysystem ).

That given I would still never allow this on my system, signed or not
signed. Also this would divert a lot of dev attention from working on the
distro proper.. Instead they would be building (and yielding TONS of emails
from users if they signed the builds as good and they are not). I know this
from experience doing this stuff with Gentoo.
IMO this is something we should still avoid for now.
If you haven't noticed I have problems with the whole concept of p2p
distribution of anything for an operating system, and I am not alone on this
stuff.

im not pushing for p2p.. Just commenting on the fact that the security
model should hold up no matter what method is used for delivery. On
binaries or on source :D.


I feel exactly the same about p2p as you do.

Post by Mark Guertin
Potentially this could make M$ email virii look tame from a security
standpoint...if one signed pkg gets compromised and distributed as trusted
(signed) into a p2p system. With p2p you have zero control after it has
gotten into nodes, that compromised build could live forever, revoking or
no. At least on (semi) controlled mirrors we can properly revoke it if the
need should ever arise.

This is a verry real thing in gentoo now, and is no different if the
trojan is in source or binary if you have no way to verify its source.
We have to assume that at some point this will happen, and have a system
that can easuly recover from it.

controll if through the keyring and keyservers. Its up to the user. If
the files are out in the wild with old invalid sigs It will not be
installed ( unless the user wants it that bad ) .. like i stated above
Im not pushing p2p but im Definatly pushing a security model that would
work in the p2p setting.

Remeber that a revocation isn't on a package, but on a key.. So even if
there out there with those bad sigs. It's still not "valid"

Post by Mark Guertin
Mark

[Will Reid]

Post by Mark Guertin

Post by jesse
If they are isgned by our devs who are on your keyring .. then you
should be able to trust. Esp if it's got 2 or more valid sigs.
Over any distrib medium a package that had this requirement would be
considered pristine. You would be able to check those sigs agains
keyservers as well as your local copy of the pubkeys.you could trust
that package if all those check were ok.
If random joe-schmo builds a binary and pops it on p2portage signed. It
doesnt give it any more credability in the eyes of porteous. ( not part
of or keysystem ).

That given I would still never allow this on my system, signed or not
signed. Also this would divert a lot of dev attention from working on the
distro proper.. Instead they would be building (and yielding TONS of emails
from users if they signed the builds as good and they are not). I know
this from experience doing this stuff with Gentoo.
IMO this is something we should still avoid for now.
If you haven't noticed I have problems with the whole concept of p2p
distribution of anything for an operating system, and I am not alone on
this stuff.
If p2p ever gets implemented I for one will want HUGE guarantees that the
option not only be able to be disabled, but that there also be a version of
the pkgmgr that this option literally does not even exist in. One user
turning on something like this unchecked can have massive repercussions on
a large LAN. It goes against the grain in corporate environment in a very
big way here.
Potentially this could make M$ email virii look tame from a security
standpoint...if one signed pkg gets compromised and distributed as trusted
(signed) into a p2p system. With p2p you have zero control after it has
gotten into nodes, that compromised build could live forever, revoking or
no. At least on (semi) controlled mirrors we can properly revoke it if the
need should ever arise.
Mark
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

Just gonna add my thoughts. I have a feeling that p2p implementation will be
so much extra design, coding, and implementation getting it to work that it
should be put off for now. And I agree there should be an option to disable
the filesharing when p2p source file sharing is implemented. Being on dialup
I suffer whenever I have to upload a file. Normally I end up lagging out,
the file stalls, and all the time was wasted anyway. My internet connection
would be virtually unusable if multiple cable users tried to download X or
KDE from me (even in part). And they'd likely never know, and I wouldn't
have fun trying to figure out why google's front page takes 5 minutes to
load.

I would REALLY like to see binary packages available. Not for every package
in the tree, but the base install (X and a few common WM also) at the least.
Also it would be nice to have different CFLAGS for them. Again, not
everything. i586, Pentium2, and Athlon for x86 would make many people happy.
You can always run `nice -n 19 emerge -e world` in a screen or VC once you're
in X and going about your thing and want "better" CFLAGS and never notice
your box was working. A 6 hour base install on the average speed box is
enough to scare a lot of users away when they find out that's without Xfree.
Basically the run down of how I think things will/should/could look soon:
Stage1 - Build it all yourself.
Stage2 - You have a compiler and a few libraries.
Stage3 - Working Non-X box (sans kernel)
Stage4 - Working X box with your choice of WM available as a seperate binary
to keep the stage tarball size down (also no precompiled kernel)
Stage5 - Quick Install, bloated, bubblie crap that people new to the distro
want to give it a quick try. X, KDE/Gnome, desktop apps a plenty, prebuilt
modularized kernel. Basically whatever the devs can stuff in a 640/700MB
ISO.

Yes, the "stage5" I recommend will be very large and use up much bandwidth,
but most users will likely avoid ever downloading it. Only those that need
it to just get their box up and running quickly to try out Zynot (or whatever
it will be called then) will pick this install method. We could even look at
something like jidgo (which debian uses to make CD-r and DVD-r ISOs) to
create the "stage5" installer CDs on the fly from multiple source servers.
The problem with jidgo is its complicated usage. Ok I'm starting to rant,
so I'll stop on this unless anyone requests more out of me. :)

This is probably getting way off-topic, but the last thing I'd like to
suggest is a GUI installer. If anyone can point me to the right place to
read up and learn how to code widgets I'd like to get involved in a gtk2
installer project that will make people think they're installing something
officially better than RedHat because it's prettier and even easier ;) to
install.

--Will (Sif/SanityInFlux - if you didn't know)

[jesse]

Post by Will Reid

Post by Mark Guertin

Post by jesse
If they are isgned by our devs who are on your keyring .. then you
should be able to trust. Esp if it's got 2 or more valid sigs.
Over any distrib medium a package that had this requirement would be
considered pristine. You would be able to check those sigs agains
keyservers as well as your local copy of the pubkeys.you could trust
that package if all those check were ok.
If random joe-schmo builds a binary and pops it on p2portage signed. It
doesnt give it any more credability in the eyes of porteous. ( not part
of or keysystem ).

That given I would still never allow this on my system, signed or not
signed. Also this would divert a lot of dev attention from working on the
distro proper.. Instead they would be building (and yielding TONS of emails
from users if they signed the builds as good and they are not). I know
this from experience doing this stuff with Gentoo.
IMO this is something we should still avoid for now.
If you haven't noticed I have problems with the whole concept of p2p
distribution of anything for an operating system, and I am not alone on
this stuff.
If p2p ever gets implemented I for one will want HUGE guarantees that the
option not only be able to be disabled, but that there also be a version of
the pkgmgr that this option literally does not even exist in. One user
turning on something like this unchecked can have massive repercussions on
a large LAN. It goes against the grain in corporate environment in a very
big way here.
Potentially this could make M$ email virii look tame from a security
standpoint...if one signed pkg gets compromised and distributed as trusted
(signed) into a p2p system. With p2p you have zero control after it has
gotten into nodes, that compromised build could live forever, revoking or
no. At least on (semi) controlled mirrors we can properly revoke it if the
need should ever arise.
Mark
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

Just gonna add my thoughts. I have a feeling that p2p implementation will be
so much extra design, coding, and implementation getting it to work that it
should be put off for now. And I agree there should be an option to disable
the filesharing when p2p source file sharing is implemented. Being on dialup
I suffer whenever I have to upload a file. Normally I end up lagging out,
the file stalls, and all the time was wasted anyway. My internet connection
would be virtually unusable if multiple cable users tried to download X or
KDE from me (even in part). And they'd likely never know, and I wouldn't
have fun trying to figure out why google's front page takes 5 minutes to
load.
I would REALLY like to see binary packages available. Not for every package
in the tree, but the base install (X and a few common WM also) at the least.
Also it would be nice to have different CFLAGS for them. Again, not
everything. i586, Pentium2, and Athlon for x86 would make many people happy.
You can always run `nice -n 19 emerge -e world` in a screen or VC once you're
in X and going about your thing and want "better" CFLAGS and never notice
your box was working. A 6 hour base install on the average speed box is
enough to scare a lot of users away when they find out that's without Xfree.
Stage1 - Build it all yourself.
Stage2 - You have a compiler and a few libraries.
Stage3 - Working Non-X box (sans kernel)
Stage4 - Working X box with your choice of WM available as a seperate binary
to keep the stage tarball size down (also no precompiled kernel)
Stage5 - Quick Install, bloated, bubblie crap that people new to the distro
want to give it a quick try. X, KDE/Gnome, desktop apps a plenty, prebuilt
modularized kernel. Basically whatever the devs can stuff in a 640/700MB
ISO.
Yes, the "stage5" I recommend will be very large and use up much bandwidth,
but most users will likely avoid ever downloading it. Only those that need
it to just get their box up and running quickly to try out Zynot (or whatever
it will be called then) will pick this install method. We could even look at
something like jidgo (which debian uses to make CD-r and DVD-r ISOs) to
create the "stage5" installer CDs on the fly from multiple source servers.
The problem with jidgo is its complicated usage. Ok I'm starting to rant,
so I'll stop on this unless anyone requests more out of me. :)
This is probably getting way off-topic, but the last thing I'd like to
suggest is a GUI installer. If anyone can point me to the right place to
read up and learn how to code widgets I'd like to get involved in a gtk2
installer project that will make people think they're installing something
officially better than RedHat because it's prettier and even easier ;) to
install.
--Will (Sif/SanityInFlux - if you didn't know)

dialog and sh make for really quick gui installers ( think redhat 5.2
6.2 )

Im doing a dialog installer for a contract right now. Its really pretty
simple to do once the helper functions are written, but unfortunatly it
wont be OSS..

Gtk/kde installers IMHO are just eyecandy.. :P

Post by Will Reid
_______________________________________________
Zynaut mailing list
http://lists.zynot.org/mailman/listinfo/zynaut

[Mark Bainter]

Post by Mark Guertin
As for stage tarballs this is another discussion we have to bring up,
because I think the way that Gentoo did things in this area was pretty messy
and I think a single stage2 (base) and some binary pkgs would be much more
suitable. This would also cut down installation and distro build times
immensely.

I'd just like to put in my vote /against/ just going with a stage 2
release. I built almost all of my systems stage1. (With only one
exception actually, and that's cause it was an original Pentium
system with a very slow disk and I didn't have time to wait.)
I'd be rather put out if we supplied only the stage2 tarballs.

--

[Mark Guertin]

Post by Mark Bainter
I'd just like to put in my vote /against/ just going with a stage 2
release. I built almost all of my systems stage1. (With only one
exception actually, and that's cause it was an original Pentium
system with a very slow disk and I didn't have time to wait.)
I'd be rather put out if we supplied only the stage2 tarballs.

A stage1 would still be available, but my concerns are tegaretted more
towards rethinking the way we do a stage3.

It has been argued very very many times that "I don't want this in my
stage3" and "I need this other tool in my stage3". I propose to get rid of
stage3, provide a stage1 as a developers type pkg (the only real need for a
stage1 is if you want to re-bootstrap your system with a newer compiler or
toolchain compoenent or to port to another arch). The pkgs in the bootstrap
configuration do not leverage USE variables in any way during the bootstrap,
so aside from these needs it is not needed as long as it's up to date.

The new 'stage3' would be much simpler if you had a simple list of optional
pkgs to install, you choose what you want and go from there. Should you
choose to go the source route it wouldn't make much of a difference, you
could still go from there to emerge things from source.

Mark

=====================================
Mark Guertin
Technology Manager, Bruce Mau Design
197 Spadina Ave, Suite 501
Toronto, ON, Canada M5T 2C8
***@brucemaudesign.com
416-260-5777 ext. 291
=====================================


This email and its attachments are confidential: they are for use of the
named recipient(s) only. Information contained in this email and attachments
to it are protected by privilege, work product immunity, copyright and other
applicable law. If you are not the intended recipient please notify us
immediately by telephone at 416.260.5777 or by e-mail at
***@brucemaudesign.com and delete this email from your files.