| Age | Commit message (Collapse) | Author |
|---|
|
* Parted: Allow partitions to have no filesystem, for eg, GPT BIOS boot
partitions. (API change)
* Added rawPartition to PartSpec, for specifying partitions with no
filesystem.
* Added BiosGrubFlag to PartFlag.
Note that man parted does not list the "bios_boot" flag, but I found it in
its html documentation. Other flags may also be missing.
This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
|
|
This is to support eg, coreboot. The GrubTarget passed to Grub.installed
is introspected to determine --target. If multiple grubs are installed,
it currently doesn't pass any --target. Might make more sense to run
grub-install repeatedly, but I don't know if that case is sane at all.
The Xen -> "x86_64-xen" mapping is kind of arbitrarily
chosen since there's a i386-xen available too. I don't know when that
case would be used in any case though; chainPVGrub uses installed Xen,
but it does not run grub-install. If this does become a problem,
would probably need to split it into Xen64 and Xen32.
Renamed BIOS to GrubTarget in passing to match grub's terminology; BIOS was
kind of a joke term for this in propellor.
This commit was sponsored by Francois Marier on Patreon.
|
|
That can take quite a while, so let the user know why propellor has
stalled.
|
|
* DiskImage: Fix rsync crash when a mount point does not exist in the
chroot.
* Fix bug in unmountBelow that caused unmounting of nested mounts to
fail.
This commit was sponsored by Jack Hill on Patreon.
|
|
Cheap flash drives need partitions aligned to 4 MiB in order to not be slow
(and to avoid extra writes). <https://lwn.net/Articles/428584/>
And at least 1 MiB alignment is generally a good idea, and most people
seem to think 4 MiB is for all drives.
I noticed that Parted.partitioned does not do that; the first
partition started at an offset of 1 MB, and subsequent partitions
from where it ends. (The 1 MB offset came from the PartedVal PartSize
instance, and note that it was not 1 MiB.)
* Parted: Add an Alignment parameter. (API change)
A good default to use is safeAlignment, which is 4MiB,
well suited for inexpensive flash drives, and fine for other disks too.
Previously, a very non-optimial 1MB (not 1MiB) alignment had been used.
* DiskImage: Use safeAlignment. It didn't seem worth making the
alignment configurable here.
Alignment is implemented by offsetting the first partition's start
position so it's aligned (making sure to leave room for the partition
table). Each partition is then extended as needed so the next partition
will start properly aligned.
Note that parted rejects partition tables that don't fit in cylinder
bounderies. Before, propellor let parted deal with the fine details
of layout, so that was not a problem. Now it's possible to set some
wacky Alignment not divisible by 512, or use Byte sizes for partitions
and create a partition table that parted rejects. But, using
safeAlignment and MegaBytes should always be safe.
Also, this fixes a rounding bug in Parted.calcPartTable.
It was rounding up to the nearest MegaByte when allocating remaining
disk space, so returned partition table that was actually larger than
the disk size.
This commit was sponsored by an anonymous bitcoiner.
|
|
|
|
Diskimage.imageBuiltFor: New property to build a disk image for a Host,
using partition table information configured via the new properties
hasPartitionTableType, hasPartition and adjustPartition.
This lets Machine properties include eg /boot partitions that are known to
be needed by the bootloader, and the user can adjust those partitions and
add others.
This commit was sponsored by Brock Spratlen on Patreon.
|
|
|
|
Reconsidered making services never run inside chroots, that seemed too
potentially limiting.
Using Info rather than checking policy-rc.d because it will also work
outside of debian, but more because policy-rc.d has an extremely
complicated interface and I didn't want to deal with it.
This commit was sponsored by Jochen Bartl on Patreon.
|
|
|
|
flashKernelMounted is slightly cargo culted from Grub.bootsMounted,
could be refactored.
This commit was sponsored by Thom May on Patreon.
|
|
This commit was sponsored by Denis Dzyubenko on Patreon.
|
|
|
|
|
|
Installing u-boot to the boot sector is not needed by some boards
(my CubieTruck boots without it), but may be by others.
Tricky part was making u-boot be written to a disk image when building one.
This commit was sponsored by Jake Vosloo on Patreon.
|
|
Can be used to create disk images for arm boards using flash-kernel.
This commit was sponsored by Ewen McNeill.
|
|
* DiskImage: Made a DiskImage type class, so that different disk image
formats can be implemented. The properties in this module can generate
any type that is a member of DiskImage. (API change)
(To convert existing configs, convert the filename of the disk image
to RawDiskImage filename.)
* Removed DiskImage.vmdkBuiltFor property. (API change)
Instead, use VirtualBoxPointer in the property that creates the disk
image.
This commit was sponsored by Jack Hill on Patreon.
|
|
|
|
|
|
|
|
|
|
|
|
* Added Rsync.installed property.
* Added DiskImage.vmdkBuilt property which is useful for booting
a disk image in VirtualBox.
|
|
|
|
This commit was sponsored by Anthony DeRobertis on Patreon.
|
|
* Generalized the PartSpec DSL, so it can be used for both
disk image partitioning, and disk device partitioning, with
different partition sizing methods as appropriate for the different
uses. (minor API change)
* Propellor.Property.Parted: Added calcPartTable function which uses
PartSpec DiskPart, and a useDiskSpace combinator.
This commit was sponsored by Thomas Hochstein on Patreon.
|
|
|
|
|
|
Properties that used to need it as a parameter now look at Info about the
bootloader that is installed in the chroot that the disk image is created
from. (API change)
This is a simplication, and avoids the user needing to repeat themselves
in the propellor config, thus avoiding mistakes.
When no boot loader is installed, or multiple different ones are,
disk image creation will fail, which seems reasonable.
This commit was sponsored by Jake Vosloo on Patreon.
|
|
* DiskImage.grubBooted no longer takes a BIOS parameter,
and no longer implicitly adds Grub.installed to the properties of
the disk image. If you used DiskImage.grubBooted, you'll need to update
your propellor configuration, removing the BIOS parameter from
grubBooted and adding a Grub.installed property to the disk image, eg:
& Grub.installed PC
(API change)
* Grub.installed: Avoid running update-grub when used in a chroot, since
it will get confused.
* DiskImage.Finalization: Simplified this type since it does not need to
be used to install packages anymore. (API change)
The advantage of doing this comes when using hostChroot with
imageBuilt, since the Host then has its Grub.installed property
explicitly listed so propellor knows about it when otherwise deploying that
host. Also, it simplifies the quite complex imageBuilt parameters.
This commit was sponsored by Ewen McNeill.
|
|
* Propellor.Property.XFCE added with some useful properties for the
desktop environment.
* Added File.applyPath property.
This commit was sponsored by Riku Voipio.
|
|
Since some programs (such as VBoxManage convertdd) refuse to operate on
disk images not aligned to a sector size.
This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
|
|
* Fix bug when using setContainerProps with a chroot that prevented
properties added to a chroot that way from being seen when propellor
was running inside the chroot. This affected disk image creation, and
possibly other things that use chroots.
The problem was, propagateChrootInfo was being passed the initial
version of the Chroot, but then the Chroot got more properties
added, and so those were not recorded in the _chroot info.
Fix was simply to make InfoPropagator be passed the Chroot as an
additional parameter, so Chroot.provisioned' can pass in the final
Chroot to it.
|
|
* DiskImage building properties used to propagate DNS info out from
the chroot used to build the disk image to the Host. That is no longer
done, since that chroot only exists as a side effect of the disk image
creation and servers will not be running in it.
* The IsInfo types class's propagateInfo function changed to use a
PropagateInfo data type. (API change)
This is particularly important when using hostChroot, since the host could
well have DNS settings then.
This commit was sponsored by Ole-Morten Duesund on Patreon.
|
|
As originally seen in my slides at Linux.Conf.Au 2017 in January.
Now that it's not vaporware, it allows one Host to build a disk image that
has all the properties of another Host.
It was easier than I thought to implement this! As expected, Info
propagation was slightly tricky. Also, I originally had a lot of machinery
to try to use Info to detect infinitely nested chroot loops. But, my
machinery didn't work, and when I tested it, ghc did a much better job,
causing a "warning: <<loop>>" message to be output instead of such a
property using infinite disk space.
This commit was sponsored by Bruno BEAUFILS on Patreon.
|
|
|
|
|
|
Signed-off-by: Zihao Wang <dev@wzhd.org>
|
|
TODO: remove ANDROID (used in GitAnnexBuilder)
TODO: add other architectures
TODO: rename ARMHF
TODO: rename ARMEL
(cherry picked from commit 6f36f6cade4e1d8b15c714565e223562c6573099)
|
|
|
|
|
|
Unfortunately, DiskImage needs to add properties to the Chroot it's
presented with, and the metatypes are not included in the Chroot, so it
can't guarantee that the properties it's adding match the OS in the Chroot.
I partially worked around this by making the properties that DiskImage adds
check the OS, so they don't assume Debian.
It would be nicer to parameterize the Chroot type with the metatypes of the
inner OS. I worked for several hours on a patch along those lines, but it
doesn't quite compile. Failed at the final hurdle :/ The patch is below
for later..
--- src/Propellor/Property/Chroot.hs 2016-03-27 16:06:44.285464820 -0400
+++ /home/joey/Chroot.hs 2016-03-27 15:32:29.073416143 -0400
@@ -1,9 +1,9 @@
-{-# LANGUAGE FlexibleContexts, GADTs, DeriveDataTypeable #-}
+{-# LANGUAGE FlexibleContexts, GADTs, DeriveDataTypeable, MultiParamTypeClasses, TypeSynonymInstances, FlexibleInstances, DataKinds #-}
module Propellor.Property.Chroot (
debootstrapped,
bootstrapped,
- provisioned,
+ --provisioned,
Chroot(..),
ChrootBootstrapper(..),
Debootstrapped(..),
@@ -11,7 +11,7 @@
noServices,
inChroot,
-- * Internal use
- provisioned',
+ --provisioned',
propagateChrootInfo,
propellChroot,
chain,
@@ -20,6 +20,7 @@
import Propellor.Base
import Propellor.Container
+import Propellor.Types.MetaTypes
import Propellor.Types.CmdLine
import Propellor.Types.Chroot
import Propellor.Types.Info
@@ -38,27 +39,29 @@
-- | Specification of a chroot. Normally you'll use `debootstrapped` or
-- `bootstrapped` to construct a Chroot value.
-data Chroot where
- Chroot :: ChrootBootstrapper b => FilePath -> b -> Host -> Chroot
-
-instance IsContainer Chroot where
- containerProperties (Chroot _ _ h) = containerProperties h
- containerInfo (Chroot _ _ h) = containerInfo h
- setContainerProperties (Chroot loc b h) ps = Chroot loc b (setContainerProperties h ps)
+--
+-- The inner and outer type variables are the metatypes of the inside of
+-- the chroot and the system it runs in.
+data Chroot inner outer where
+ Chroot :: ChrootBootstrapper b inner outer => FilePath -> b -> Host -> (inner, outer) -> Chroot inner outer
+
+instance IsContainer (Chroot inner outer) where
+ containerProperties (Chroot _ _ h _) = containerProperties h
+ containerInfo (Chroot _ _ h _) = containerInfo h
-chrootSystem :: Chroot -> Maybe System
+chrootSystem :: Chroot inner outer -> Maybe System
chrootSystem = fromInfoVal . fromInfo . containerInfo
-instance Show Chroot where
- show c@(Chroot loc _ _) = "Chroot " ++ loc ++ " " ++ show (chrootSystem c)
+instance Show (Chroot inner outer) where
+ show c@(Chroot loc _ _ _) = "Chroot " ++ loc ++ " " ++ show (chrootSystem c)
-- | Class of things that can do initial bootstrapping of an operating
-- System in a chroot.
-class ChrootBootstrapper b where
+class ChrootBootstrapper b inner outer where
-- | Do initial bootstrapping of an operating system in a chroot.
-- If the operating System is not supported, return
-- Left error message.
- buildchroot :: b -> Maybe System -> FilePath -> Either String (Property Linux)
+ buildchroot :: b -> Maybe System -> FilePath -> Either String (Property outer)
-- | Use this to bootstrap a chroot by extracting a tarball.
--
@@ -68,9 +71,8 @@
-- detect automatically.
data ChrootTarball = ChrootTarball FilePath
-instance ChrootBootstrapper ChrootTarball where
- buildchroot (ChrootTarball tb) _ loc = Right $
- tightenTargets $ extractTarball loc tb
+instance ChrootBootstrapper ChrootTarball UnixLike UnixLike where
+ buildchroot (ChrootTarball tb) _ loc = Right $ extractTarball loc tb
extractTarball :: FilePath -> FilePath -> Property UnixLike
extractTarball target src = check (unpopulated target) $
@@ -88,7 +90,7 @@
-- | Use this to bootstrap a chroot with debootstrap.
data Debootstrapped = Debootstrapped Debootstrap.DebootstrapConfig
-instance ChrootBootstrapper Debootstrapped where
+instance ChrootBootstrapper Debootstrapped DebianLike Linux where
buildchroot (Debootstrapped cf) system loc = case system of
(Just s@(System (Debian _) _)) -> Right $ debootstrap s
(Just s@(System (Buntish _) _)) -> Right $ debootstrap s
@@ -107,13 +109,22 @@
-- > & osDebian Unstable "amd64"
-- > & Apt.installed ["ghc", "haskell-platform"]
-- > & ...
-debootstrapped :: Debootstrap.DebootstrapConfig -> FilePath -> Chroot
+-- debootstrapped :: Debootstrap.DebootstrapConfig -> FilePath -> Chroot DebianLike
+debootstrapped
+ :: (SingI inner, SingI outer, ChrootBootstrapper Debootstrapped (MetaTypes inner) (MetaTypes outer))
+ => Debootstrap.DebootstrapConfig
+ -> FilePath
+ -> Chroot (MetaTypes inner) (MetaTypes outer)
debootstrapped conf = bootstrapped (Debootstrapped conf)
-- | Defines a Chroot at the given location, bootstrapped with the
-- specified ChrootBootstrapper.
-bootstrapped :: ChrootBootstrapper b => b -> FilePath -> Chroot
-bootstrapped bootstrapper location = Chroot location bootstrapper h
+bootstrapped
+ :: (SingI inner, SingI outer, ChrootBootstrapper b (MetaTypes inner) (MetaTypes outer))
+ => b
+ -> FilePath
+ -> Chroot (MetaTypes inner) (MetaTypes outer)
+bootstrapped bootstrapper location = Chroot location bootstrapper h (sing, sing)
where
h = Host location [] mempty
@@ -123,45 +134,79 @@
-- Reverting this property removes the chroot. Anything mounted inside it
-- is first unmounted. Note that it does not ensure that any processes
-- that might be running inside the chroot are stopped.
-provisioned :: Chroot -> RevertableProperty (HasInfo + Linux) Linux
+-- provisioned :: SingI outer => Chroot inner outer -> RevertableProperty (HasInfo + MetaTypes outer) Linux
+provisioned
+ ::
+ ( SingI outer
+ , SingI metatypes
+ , Combines (Property (MetaTypes outer)) (Property (MetaTypes outer))
+ , (HasInfo + outer) ~ MetaTypes metatypes
+ , CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer)) ~ Property outer
+ , IncludesInfo (MetaTypes metatypes) ~ 'True)
+ => Chroot inner outer -> RevertableProperty (HasInfo + outer) Linux
provisioned c = provisioned' (propagateChrootInfo c) c False
provisioned'
- :: (Property Linux -> Property (HasInfo + Linux))
- -> Chroot
+ ::
+ ( Combines (Property (MetaTypes outer)) (Property (MetaTypes outer))
+ , CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer)) ~ Property outer
+ , SingI outer
+ )
+ => (Property outer -> Property (HasInfo + outer))
+ -> Chroot inner outer
-> Bool
- -> RevertableProperty (HasInfo + Linux) Linux
-provisioned' propigator c@(Chroot loc bootstrapper _) systemdonly =
- (propigator $ setup `describe` chrootDesc c "exists")
+ -> RevertableProperty (HasInfo + outer) Linux
+provisioned' propigator c systemdonly =
+ (propigator $ setup c systemdonly `describe` chrootDesc c "exists")
<!>
- (teardown `describe` chrootDesc c "removed")
- where
- setup :: Property Linux
- setup = propellChroot c (inChrootProcess (not systemdonly) c) systemdonly
- `requires` built
-
- built = case buildchroot bootstrapper (chrootSystem c) loc of
- Right p -> p
- Left e -> cantbuild e
-
- cantbuild e = property (chrootDesc c "built") (error e)
-
- teardown :: Property Linux
- teardown = check (not <$> unpopulated loc) $
- property ("removed " ++ loc) $
- makeChange (removeChroot loc)
-
-propagateChrootInfo :: Chroot -> Property Linux -> Property (HasInfo + Linux)
-propagateChrootInfo c@(Chroot location _ _) p = propagateContainer location c $
- p `addInfoProperty` chrootInfo c
+ (teardown c `describe` chrootDesc c "removed")
-chrootInfo :: Chroot -> Info
-chrootInfo (Chroot loc _ h) = mempty `addInfo`
+-- chroot removal code is currently linux specific..
+teardown :: Chroot inner outer -> Property Linux
+teardown (Chroot loc _ _ _) = check (not <$> unpopulated loc) $
+ property ("removed " ++ loc) $
+ makeChange (removeChroot loc)
+
+setup
+ ::
+ ( SingI outer
+ , Combines (Property (MetaTypes outer)) (Property (MetaTypes outer))
+ )
+ => Chroot inner outer
+ -> Bool
+ -> CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer))
+setup c systemdonly = propellChroot c (inChrootProcess (not systemdonly) c) systemdonly
+ `requires` built c
+
+built :: (SingI outer, ChrootBootstrapper b inner outer) => Chroot inner outer -> Property (MetaTypes outer)
+built c@(Chroot loc bootstrapper _ _) =
+ case buildchroot bootstrapper (chrootSystem c) loc of
+ Right p -> error "FOO" -- p
+ Left e -> error "FOO" -- cantbuild c e
+
+cantbuild :: Chroot inner outer -> String -> Property UnixLike
+cantbuild c e = property (chrootDesc c "built") (error e)
+
+propagateChrootInfo
+ ::
+ ( SingI metatypes
+ , (HasInfo + outer) ~ MetaTypes metatypes
+ , IncludesInfo (MetaTypes metatypes) ~ 'True
+ )
+ => Chroot inner outer
+ -> Property outer
+ -> Property (MetaTypes metatypes)
+propagateChrootInfo c@(Chroot location _ _ _) p =
+ propagateContainer location c $
+ p `addInfoProperty` chrootInfo c
+
+chrootInfo :: Chroot inner outer -> Info
+chrootInfo (Chroot loc _ h _) = mempty `addInfo`
mempty { _chroots = M.singleton loc h }
-- | Propellor is run inside the chroot to provision it.
-propellChroot :: Chroot -> ([String] -> IO (CreateProcess, IO ())) -> Bool -> Property UnixLike
-propellChroot c@(Chroot loc _ _) mkproc systemdonly = property (chrootDesc c "provisioned") $ do
+propellChroot :: SingI outer => Chroot inner outer -> ([String] -> IO (CreateProcess, IO ())) -> Bool -> Property (MetaTypes outer)
+propellChroot c@(Chroot loc _ _ _) mkproc systemdonly = property (chrootDesc c "provisioned") $ do
let d = localdir </> shimdir c
let me = localdir </> "propellor"
shim <- liftIO $ ifM (doesDirectoryExist d)
@@ -199,8 +244,8 @@
liftIO cleanup
return r
-toChain :: HostName -> Chroot -> Bool -> IO CmdLine
-toChain parenthost (Chroot loc _ _) systemdonly = do
+toChain :: HostName -> Chroot inner outer -> Bool -> IO CmdLine
+toChain parenthost (Chroot loc _ _ _) systemdonly = do
onconsole <- isConsole <$> getMessageHandle
return $ ChrootChain parenthost loc systemdonly onconsole
@@ -224,8 +269,8 @@
putStrLn $ "\n" ++ show r
chain _ _ = errorMessage "bad chain command"
-inChrootProcess :: Bool -> Chroot -> [String] -> IO (CreateProcess, IO ())
-inChrootProcess keepprocmounted (Chroot loc _ _) cmd = do
+inChrootProcess :: Bool -> Chroot inner outer -> [String] -> IO (CreateProcess, IO ())
+inChrootProcess keepprocmounted (Chroot loc _ _ _) cmd = do
mountproc
return (proc "chroot" (loc:cmd), cleanup)
where
@@ -244,26 +289,24 @@
provisioningLock :: FilePath -> FilePath
provisioningLock containerloc = "chroot" </> mungeloc containerloc ++ ".lock"
-shimdir :: Chroot -> FilePath
-shimdir (Chroot loc _ _) = "chroot" </> mungeloc loc ++ ".shim"
+shimdir :: Chroot inner outer -> FilePath
+shimdir (Chroot loc _ _ _) = "chroot" </> mungeloc loc ++ ".shim"
mungeloc :: FilePath -> String
mungeloc = replace "/" "_"
-chrootDesc :: Chroot -> String -> String
-chrootDesc (Chroot loc _ _) desc = "chroot " ++ loc ++ " " ++ desc
+chrootDesc :: Chroot inner outer -> String -> String
+chrootDesc (Chroot loc _ _ _) desc = "chroot " ++ loc ++ " " ++ desc
|
|
Also, implemented modifyHostProps to add properties to an existing host.
Using it bypasses some type safety. Its use in docker is safe though.
But, in Conductor, the use of it was not really safe, because it was used
with a DebianLike property. Fixed that by making Ssh.installed
target all unix's, although it will fail on non-DebianLike ones.
|
|
|
|
* Properties that run an arbitrary command, such as cmdProperty
and scriptProperty are converted to use UncheckedProperty, since
they cannot tell on their own if the command truely made a change or not.
(API Change)
Transition guide:
- When GHC complains about an UncheckedProperty, add:
`assume` MadeChange
- Since these properties used to always return MadeChange, that
change is always safe to make.
- Or, if you know that the command should modifiy a file, use:
`changesFile` filename
* A few properties have had their Result improved, for example
Apt.buldDep and Apt.autoRemove now check if a change was made or not.
|
|
|
|
|
|
RevertableProperty used to be assumed to contain info, but this is now made
explicit, with RevertableProperty HasInfo or RevertableProperty NoInfo.
Transition guide:
- If you define a RevertableProperty, expect some type check
failures like: "Expecting one more argument to ‘RevertableProperty’".
- Change it to "RevertableProperty NoInfo"
- The compiler will then tell you if it needs "HasInfo" instead.
- If you have code that uses the RevertableProperty constructor
that fails to type check, use the more powerful <!> operator
|
|
|
|
|
