λ©°μΉ μ μ 12-stableλ‘ μ
κ·Έλ μ΄λν ν κ°μ₯μμ poudriereλ₯Ό μ€ννλ λ° λ¬Έμ κ° μμ΅λλ€. λ΄ κ΅¬μ±μ λ³κ²½νμ§ μκ³ λΉλ μ€μ poudriere builder κ°μ₯μ΄ μμλλ λμ μ΄μ jail: IPv4 addresses clash
λ₯Ό μ»μ΅λλ€.
# poudriere bulk -j 12amd64 -p local -z zs64 -f /root/11amd64-local-zs64-pkglist
[00:00:00] Creating the reference jail... done
[00:00:01] Mounting system devices for 12amd64-local-zs64
[00:00:01] Mounting ports/packages/distfiles
[00:00:01] Using packages from previously failed build
[00:00:01] Mounting ccache from: /var/cache/ccache
[00:00:01] Mounting packages from: /p/data/packages/12amd64-local-zs64
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/zs64-make.conf
/etc/resolv.conf -> /p/data/.m/12amd64-local-zs64/ref/etc/resolv.conf
[00:00:02] Starting jail 12amd64-local-zs64
jail: IPv4 addresses clash
[00:00:02] Cleaning up
[00:00:02] Unmounting file systems
λλ LOIP4 λ° LOIP6κ³Ό μΈλΆ κ°μ₯ IP ꡬμ±μ λ§μ§μκ±°λ Έμ§λ§ μ무 μμ©μ΄ μμμ΅λλ€.
common.sh
μ μ½λλ₯Ό κ²μ¬νλ©΄ poudriereκ° ip4.addr
λ° ip6.addr
μ κ°μ΄ κ°μ₯ μμ νΈμΆμ μ λ¬ν IPλ₯Ό κ²°μ νλ €κ³ μλνλ κ²μΌλ‘ 보μ
λλ€. localipargs
λ₯Ό λΉ λ¬Έμμ΄λ‘ νλ 리μ
νμ΅λλ€(7734λ²μ§Έ μ€μμ case $IPS
λΈλ‘ λ°λ‘ λ€). κ·Έλ¬λ©΄ λΉλκ° λ€μ μ¬λ°λ₯΄κ² μλν©λλ€.
μ΄κ²μ 12-stableκ³Ό μ μ¬μ μΈ λΉνΈνμ±μ λκΉ?
μ μ₯, κ·Έ ν¨μΉμλ λΆκ΅¬νκ³ Pythonκ³Ό Rubyλ λΉλμ μ€ν¨ν©λλ€.
νμ΄μ¬:
checking getaddrinfo bug... yes
Fatal: You must get working getaddrinfo() function.
or you can specify "--disable-ipv6".
===> Script "configure" failed unexpectedly.
루λΉ:
compiling raddrinfo.c
raddrinfo.c:214:17: warning: implicit declaration of function 'parse_numeric_port' is invalid in C99 [-Wimplicit-function-declaration]
if (node && parse_numeric_port(service, &port)) {
^
1 warning generated.
compiling ifaddr.c
compiling getaddrinfo.c
In file included from getaddrinfo.c:86:
./addrinfo.h:165:12: error: conflicting types for 'getnameinfo'
extern int getnameinfo __P((
^
/usr/include/netdb.h:251:6: note: previous declaration is here
int getnameinfo(const struct sockaddr *, socklen_t, char *,
^
getaddrinfo.c:408:7: warning: add explicit braces to avoid dangling else [-Wdangling-else]
} else if (strcmp(sp->s_proto, "tcp") == 0) {
^
1 warning and 1 error generated.
*** Error code 1
VIMAGEμ vnet κ°μ₯μ μ¬μ©νμ¬ λ€μ μμ μ€μ μ μ»μ μ μμμ΅λλ€. "ν΄λμ" 곡μ IP μ€μ μ΄ μλνλλ‘ λ§λ€ μ μλ€λ©΄ λ°©λ²μ λ£κ³ μΆμ΅λλ€.
12.0-RELEASE-p2 κ°μ₯μμ 루λΉλ₯Ό λΉλνλ κ²μ μ¬μ ν ββμ€ν¨ν©λλ€.
11.2-RELEASE-p8 κ°μ₯ λ΄λΆμμ 루λΉλ₯Ό λΉλνλ κ²μ΄ μλν©λλ€.
μ΄λ€ ν΅μ°°λ ₯μ΄ μμ΅λκΉ?
첨λΆλ λΉλ λ‘κ·Έλ λ€μκ³Ό κ°μ΅λλ€.
ruby-2.5.3_1,1.log
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231773 μ²λΌ 보μ λλ€.
@stblassitude μλνλ €λ©΄ 무μμ μμ ν΄μΌ ν©λκΉ? νΉν vnet
μ΅μ
μ΄ νμν μμΉλ 무μμ
λκΉ? κ°μ₯μμ Poudriereλ Poudriereκ° λ§λ κ°μ₯ μμ μμ΅λκΉ?
ν΄κ²° λ°©λ²μ κ³ μ ν 루νλ°± μ£Όμλ₯Ό μ€μ νλ κ²μ λλ€.
LOIP4=127.0.0.2
LOIP6=::2
@bdrewery , μ μν μ€μ μ μ¬μ©νλ©΄ λμΌν κ²½κ³ λ° μ€λ₯κ° μμ±λ©λλ€.
(LOIP4="127.0.0.2/32"λ μλνμ΅λλ€). LOIP4κ° λ¬΄μλλ κ² κ°μ΅λλ€. λ€λ₯Έ κ³³μ μ€μ ν΄μΌ ν©λκΉ?
poudriere κ°μ₯μ 루νλ°±μ μΈλΆμ μΌλ‘ μ€μ νλ κ²λ μ€ν¨ν©λλ€.
LOIP4=127.0.0.2
LOIP6=::2
NO_ZFS=μ
FREEBSD_HOST= ftp://ftp.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/p
USE_PORTLINT=μλμ
USE_TMPFS=μλμ
DISTFILES_CACHE=/usr/ports/distfiles
NOLINUX=μ
ALLOW_MAKE_JOBS=μ
[00:00:00] Warning: No loopback address defined, consider setting LOIP6/LOIP4 or assigning a loopback address to the jail.
[00:00:00] Updating portstree "default" with portsnap...Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
In file included from getaddrinfo.c:86:
./addrinfo.h:165:12: error: conflicting types for 'getnameinfo'
extern int getnameinfo __P((
^
/usr/include/netdb.h:251:6: note: previous declaration is here
int getnameinfo(const struct sockaddr *, socklen_t, char *,
^
--- ext/cgi/escape/all ---
--- escape.o ---
compiling escape.c
--- ext/ripper/all ---
--- pre-install-rb-default ---
installing default ripper libraries
--- ../../.ext/common/ripper.rb ---
--- ext/json/all ---
--- ../../.ext/common/json/add/rational.rb ---
--- ext/fiddle/all ---
--- fiddle.o ---
--- ext/openssl/all ---
--- ossl.o ---
--- ext/fiddle/all ---
compiling fiddle.c
--- ext/openssl/all ---
compiling ossl.c
--- ext/rbconfig/sizeof/all ---
--- limits.o ---
compiling limits.c
--- ext/socket/all ---
getaddrinfo.c:408:7: warning: add explicit braces to avoid dangling else [-Wdangling-else]
} else if (strcmp(sp->s_proto, "tcp") == 0) {
^
--- ext/bigdecimal/all ---
--- ../../.ext/common/bigdecimal/math.rb ---
--- ext/socket/all ---
1 warning and 1 error generated.
*** [getaddrinfo.o] Error code 1
@bdrewery , μ μν μ€μ μ μ¬μ©νλ©΄ λμΌν κ²½κ³ λ° μ€λ₯κ° μμ±λ©λλ€.
(LOIP4="127.0.0.2/32"λ μλνμ΅λλ€). LOIP4κ° λ¬΄μλλ κ² κ°μ΅λλ€. λ€λ₯Έ κ³³μ μ€μ ν΄μΌ ν©λκΉ?
common.shμ λ€μ μ½λ λλ¬Έμ λλ€.
# If in a nested jail we may not even have a loopback to use.
if [ ${JAILED} -eq 1 ]; then
# !! Note these exit statuses are inverted
ifconfig | \
awk -vip="${LOIP6}" '$1 == "inet6" && $2 == ip {exit 1}' && \
LOIP6=
ifconfig | \
awk -vip="${LOIP4}" '$1 == "inet" && $2 == ip {exit 1}' && \
LOIP4=
fi
μ§κΈμ μ κ±°ν΄ λ³΄μΈμ. νμ¬ κ°μ₯μ μ€μ²©λ κ°μ₯μ ν λΉνλ €λ IPκ° μλμ§ νμΈνλ €κ³ ν©λλ€. (λ¬Έμ μ νκ·.)
μ£Όμ μ²λ¦¬νλ©΄ λ€μμ΄ νμλκ³ λΉλκ° μ§νλμ§ μμ΅λλ€.
jail: jail_set: Operation not permitted
νμΈμ μν΄ μ λ FreeBSD 12.0-RELEASE-p3 GENERICμ μ€ννλ νΈμ€νΈμ iocage κ°μ₯μμ Poudriereλ₯Ό μ€ννκ³ μμ΅λλ€.
# iocage list -l
+------+------------+------+-------+------+-----------------+--------------------+-----+----------+----------+
| JID | NAME | BOOT | STATE | TYPE | RELEASE | IP4 | IP6 | TEMPLATE | BASEJAIL |
+======+============+======+=======+======+=================+====================+=====+==========+==========+
| 2963 | poudriere | on | up | jail | 12.0-RELEASE-p3 | lo1|192.168.250.10 | - | - | no |
μλ λμ΄λ κ°μ₯μ λν λ£¨λΉ μ»΄νμΌ μ€ν¨:
root<strong i="12">@poudriere</strong>:~ # poudriere jail -l
[00:00:00] Warning: No loopback address defined, consider setting LOIP6/LOIP4 or assigning a loopback address to the jail.
JAILNAME VERSION ARCH METHOD TIMESTAMP PATH
11_2 11.2-RELEASE-p9 amd64 ftp 2019-03-05 13:49:56 /p/jails/11_2
12_0 12.0-RELEASE-p3 amd64 ftp 2019-03-05 13:52:05 /p/jails/12_0
@stblassitude μλνλ €λ©΄ 무μμ μμ ν΄μΌ ν©λκΉ? νΉν
vnet
μ΅μ μ΄ νμν μμΉλ 무μμ λκΉ? κ°μ₯μμ Poudriereλ Poudriereκ° λ§λ κ°μ₯ μμ μμ΅λκΉ?
λ΄ jail.conf
λ€μκ³Ό κ°μ΅λλ€.
mount.devfs;
devfs_ruleset = 4;
exec.clean;
exec.jail_user = "root";
exec.system_user = "root";
exec.consolelog = "/var/log/jail_${name}.log";
exec.prestart = "/root/bin/jail-helper prestart ${name} ${host.hostname} ${path}";
exec.poststop = "/root/bin/jail-helper poststop ${name} ${host.hostname} ${path}";
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
interface = "bridge0";
allow.raw_sockets;
allow.sysvipc;
...
pkg {
path = "/jail/pkg.zs64.net";
vnet;
vnet.interface = "ji${name}";
host.hostname = "pkg.zs64.net";
children.max = 40;
exec.prestart += "/root/bin/jail-helper prevnet ${name} bridge1";
exec.poststart += "zfs jail $name data/jail/${host.hostname}/poudriere";
exec.poststart += "jexec $name zfs mount -a";
#exec.prestop += "zfs unjail $name data/jail/${host.hostname}/poudriere";
exec.poststop += "/root/bin/jail-helper postvnet ${name}";
allow.mount;
allow.mount.devfs;
allow.mount.linprocfs;
allow.mount.nullfs;
allow.mount.procfs;
allow.mount.tmpfs;
allow.mount.zfs;
allow.socket_af;
allow.chflags;
enforce_statfs=1;
}
λμ°λ―Έ μ€ν¬λ¦½νΈλ λ€μμ μνν©λλ€.
#!/bin/sh
#
# Mount system directories via nullfs
#
cmd="$1"
name="$2"
host="$3"
path="$4"
rofs="/bin /lib /libexec /sbin /usr/bin /usr/include /usr/lib /usr/libdata /usr/libexec /usr/sbin /usr/share /usr/ports"
mountall() {
mount -t devfs -o ruleset=10 devfs ${path}/dev
ls ${path}/dev/null >/dev/null
for i in $rofs; do
mount -t nullfs -o ro "${i}" "${path}${i}"
done
mount -t nullfs -w /freebsd/distfiles ${path}/var/ports/distfiles
}
umountall() {
mount | \
sed -nEe 's#.* on ('"${path}"'/[^ ]*) \(.*#\1#p' | \
sort -r | \
xargs -n1 umount
}
vnet_create() {
ifname="$(/sbin/ifconfig epair create)"
ifbase="${ifname%%a}"
/sbin/ifconfig "${ifbase}a" up name "jo${name}" >/dev/null
/sbin/ifconfig "${ifbase}b" name "ji${name}" >/dev/null
/sbin/ifconfig "${host}" addm "jo${name}"
}
vnet_destroy() {
/sbin/ifconfig "jo${name}" destroy 2>/dev/null || true
}
case ${cmd} in
prestart)
umountall
mountall
;;
prevnet)
vnet_destroy
vnet_create
;;
poststop)
umountall
;;
postvnet)
vnet_destroy
;;
esac
pkg
κ°μ₯ λ΄λΆμμ λ€μμ μ¬μ©ν©λλ€. poudriere.conf
:
BASEFS=/usr/local/poudriere
CCACHE_DIR=/var/cache/ccache
DISTFILES_CACHE=/var/ports/distfiles
FREEBSD_HOST=https://download.FreeBSD.org
PKG_REPO_SIGNING_KEY=/usr/local/etc/poudriere.key
TMPFS_LIMIT=6
URL_BASE=http://pkg.example.com
ZPOOL=data
ZROOTFS=/jail/pkg.example.com/poudriere
ZFS λ°μ΄ν° μΈνΈλ κ°μ₯ λ΄λΆμμ λ€μκ³Ό κ°μ΅λλ€.
$ zfs list
NAME USED AVAIL REFER MOUNTPOINT
data 1.74T 858G 88K /data
data/jail 1.33T 858G 96K /jail
data/jail/pkg.example.com 34.2G 858G 5.54G /jail/pkg.example.com
data/jail/pkg.example.com/poudriere 24.3G 858G 88K /p
data/jail/pkg.example.com/poudriere/data 19.1G 858G 96K /p/data
data/jail/pkg.example.com/poudriere/data/.m 11.6G 858G 112K /p/data/.m
data/jail/pkg.example.com/poudriere/data/cache 260M 858G 23.8M /p/data/cache
data/jail/pkg.example.com/poudriere/data/logs 3.17G 858G 2.74G /p/data/logs
data/jail/pkg.example.com/poudriere/data/packages 4.12G 858G 1.95G /p/data/packages
data/jail/pkg.example.com/poudriere/data/wrkdirs 88K 858G 88K /p/data/wrkdirs
data/jail/pkg.example.com/poudriere/jails 1.27G 858G 88K /p/jails
data/jail/pkg.example.com/poudriere/jails/11amd64 88K 858G 88K /usr/local/poudriere/jails/11amd64
data/jail/pkg.example.com/poudriere/jails/12amd64 1.27G 858G 1.24G /usr/local/poudriere/jails/12amd64
data/jail/pkg.example.com/poudriere/ports 3.91G 858G 88K /p/ports
data/jail/pkg.example.com/poudriere/ports/local 3.91G 858G 903M /usr/local/poudriere/ports/local
μ λ μ΄ λ¬Έμ κ° μμλ κ² κ°μλ°, 12μΌμ μ²μ μλνκΈ° λλ¬Έμ 12 μ κ·Έλ μ΄λμ κ΄λ ¨μ΄ μλ€κ³ μκ°νμ§ μμμ΅λλ€. μ¬κΈ°μμ μ¬λ¬λΆ λͺ¨λκ° μ΄λ€ μ루μ μ μκ°ν΄ λλμ§ κ³μ μ΄ν΄λ³΄μμμ€.
ν
μ€νΈ μΌμ΄μ€λ₯Ό μ°Ύμμ΅λλ€. IPv6 μ΅μ
κ³Ό μ΄μ€ μ€ν κ°μ₯μΌλ‘ lang/python27
λλ lang/python3
λ₯Ό λΉλν©λλ€.
poudriere jail export jail.confμΈ κ²½μ°
ip4=inherit;
ip6=inherit;
λΉλ μ±κ³΅
ip4.addr
λ΄λ³΄λ΄κΈ° λ° ip6.addr
μ£Όμ λΉλκ° κ΅¬μ± μ μ€ν¨νλ κ²½μ°
checking for getaddrinfo... yes
checking getaddrinfo bug... yes
Fatal: You must get working getaddrinfo() function.
or you can specify "--disable-ipv6".
λΉνμ±ν μ΅μ
IPv6 μμ λΉλ.
λΆλͺ ν IPv6μ λν set lo μ΄λκ°μ μ€λ₯κ°μλ κ² κ°μ΅λλ€. μ΄κ²μ μ§κΈ (::1/128) /8μ΄ μλλλ€.
ν₯λ―Έλ‘κ²λ μ°¨μ΄μ μ λνν μμ
μκ° μλλλ€.
bulk -i
μλ λΉλ μ€ν¨ ꡬμ±μ μ¬μ©νλ κ²½μ° μ΄ κ°μ₯μμ μλ λΉλ μ±κ³΅
make -C /usr/ports/lang/python27/
....
checking for getaddrinfo... yes
checking getaddrinfo bug... no
checking for getnameinfo... (cached) yes
....
creating Makefile
....
RESTRICT_NETWORKING=no
μ΄ λ¬Έμ λ₯Ό μμ νκ³ NO_RESTRICT_NETWORKING_PACKAGES="python27 ..."
μ μ¬ν ALLOW_NETWORKING_PACKAGES
νμκ² μ΅λκΉ?
VIMAGEμ vnet κ°μ₯μ μ¬μ©νμ¬ λ€μ μμ μ€μ μ μ»μ μ μμμ΅λλ€. "ν΄λμ" 곡μ IP μ€μ μ΄ μλνλλ‘ λ§λ€ μ μλ€λ©΄ λ°©λ²μ λ£κ³ μΆμ΅λλ€.
μμ μ견μ λ°νμΌλ‘ vnet
μ루μ
λ μ¬μ©νμ΅λλ€. λλ₯Ό μν΄ μΌνλ€. κ°μ¬ν©λλ€.
@stblassitude μλνλ €λ©΄ 무μμ μμ ν΄μΌ ν©λκΉ? νΉν
vnet
μ΅μ μ΄ νμν μμΉλ 무μμ λκΉ? κ°μ₯μμ Poudriereλ Poudriereκ° λ§λ κ°μ₯ μμ μμ΅λκΉ?
κ·Έκ²μ poudriere κ°μ₯μ λν ꡬμ±μ λλ€. μ λ μ΄κ²μ νμ΄μ.
/etc/rc.confμ κ²½μ°:
cloned_interfaces="bridge0"
ifconfig_bridge0="addm ix0 up"
κ·Έλ° λ€μ κ°μ₯ ꡬμ±μ κ²½μ°:
$ sudo iocage set vnet=on pkg01
$ sudo iocage set ip4_addr="vnet0|10.55.0.29/24" pkg01
$ sudo iocage set ip6_addr="vnet0|[redacted]:23/64" pkg01
λ€μ μ£Ό μ λμ λΈλ‘κ·Έ κ²μλ¬Όμ΄ μμ κ²μ λλ€.
μ΄ λ΄μ©μ μ½μ ν 12.1-RELEASE-p7μ μ€ννλ λμ μ λ°μ΄νΈλ₯Ό μννκ³ μΆμμ΅λλ€.
"κ·Έκ²μ λν κ°μ₯μ κ°ν λ°λͺ¬μ΄ "루νλ°± μ£Όμ"μ ν¬νΈμ μ°κ²°λ λ μ€μ λ‘ κ°μ₯μ μΈλΆ IP μ£Όμμ μ°κ²°λλ€λ κ²μ μλ―Έν©λλ€."
Lucas, Michael W. FreeBSD λ§μ€ν°λ¦¬: κ°μ₯(IT λ§μ€ν°λ¦¬ λΆ 15)
poudriere.confμ LOIP4λ₯Ό κ°μ₯μ ip4_addrλ‘ μ€μ νλ©΄ 루λΉκ° μ»΄νμΌλ μ μμ΅λλ€.
λμ κ²½μ°μλ:
# iocageλ ip4_addr poudriereλ₯Ό μ»μ΅λλ€.
lo1|192.168.10.100
# grep LOIP4 /usr/local/etc/poudriere.conf
LOIP4=192.168.10.100
κ°μ₯ μ μ©ν λκΈ
VIMAGEμ vnet κ°μ₯μ μ¬μ©νμ¬ λ€μ μμ μ€μ μ μ»μ μ μμμ΅λλ€. "ν΄λμ" 곡μ IP μ€μ μ΄ μλνλλ‘ λ§λ€ μ μλ€λ©΄ λ°©λ²μ λ£κ³ μΆμ΅λλ€.