another attempt at Y2038

[John Paul Adrian Glaubitz]

Hi Helmut!

I was also wondering about this Y2038 thingy and did some experiments.
I'm reporting what I found to document it, but don't see much actionable
stuff here. Many thanks to Arnd Bergmann for his input.
Attempt #1: rebootstrap
Given that I develop rebootstrap, attempting to use it for a time64
bootstrap seemed quite natural. I've been talking to this with Steve
multiple times including DC22. The question was how to plug it in. In
the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to
modify dpkg-buildflags. Not every package uses these flags, but a
majority do. For a survey, this is probably good enough.

I would love to do that for m68k as well. We could use this opportunity to
rebuild the m68k port with 32-bit alignment which would solve quite a number
of problems since many projects like LLVM and Qt assume a minimum alignment
of 32 bits while m68k still defaults to 16 bits.

Since the time64 rebootstrap would break the ABI anyway, we could use to fix
alignment issue on m68k once and for all :-).

Adrian

[Steve McIntyre]

Post by John Paul Adrian Glaubitz
Hi Helmut!

I was also wondering about this Y2038 thingy and did some experiments.
I'm reporting what I found to document it, but don't see much actionable
stuff here. Many thanks to Arnd Bergmann for his input.
Attempt #1: rebootstrap
Given that I develop rebootstrap, attempting to use it for a time64
bootstrap seemed quite natural. I've been talking to this with Steve
multiple times including DC22. The question was how to plug it in. In
the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to
modify dpkg-buildflags. Not every package uses these flags, but a
majority do. For a survey, this is probably good enough.

I would love to do that for m68k as well. We could use this opportunity to
rebuild the m68k port with 32-bit alignment which would solve quite a number
of problems since many projects like LLVM and Qt assume a minimum alignment
of 32 bits while m68k still defaults to 16 bits.
Since the time64 rebootstrap would break the ABI anyway, we could use to fix
alignment issue on m68k once and for all :-).

So the reason that we're talking about doing a replacement armhf port
is for two reasons:

* it's probably one of the few 32-bit arches that is likely to still
be in routine use beyond 2038 (*many* embedded users depending on
Debian stuff)

* it's feasible to rebootstrap and break ABI as there isn't a large
corpus of older-ABI binaries that people will care about supporting
in the future. (This rules out i386 - the main reason to keep it
around is just for the older binaries AFAICS.)

I don't want to be rude, but I really don't see how m68k fits here. Of
course, feel free to do a rebootstrap if you like, but I genuinely
don't see any great need for it.

[John Paul Adrian Glaubitz]

Post by Steve McIntyre
I don't want to be rude, but I really don't see how m68k fits here. Of
course, feel free to do a rebootstrap if you like, but I genuinely
don't see any great need for it.

I don't see the need for starting another "Why does Debian Ports exist?"
discussion again. If you want to discuss about what's needed, probably,
a lot of open source projects would just have to go away. ¯\_(ツ)_/¯

People just enjoy hacking code. There doesn't always have to have a commercial
interest and a bigger reason behind it and that's perfectly fine.

Adrian

[Thomas Schmitt]

Hi,

likely to still be in routine use beyond 2038

Sidenote towards ISO 9660 image producers:

Don't forget to check from time to time whether Linux removed the
int bottleneck in fs/isofs/.
See:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38

The bug report was originally about a Y2028 rollover caused by 8-bit
signedness. It was closed after
https://github.com/torvalds/linux/commit/34be4dbf87fc
which left the int problem for future adventures.

Currently it's still "int iso_date()" in:
https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19


Have a nice day :)

Thomas

[Arnd Bergmann]

Post by Thomas Schmitt
Hi,

likely to still be in routine use beyond 2038

Don't forget to check from time to time whether Linux removed the
int bottleneck in fs/isofs/.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
The bug report was originally about a Y2028 rollover caused by 8-bit
signedness. It was closed after
https://github.com/torvalds/linux/commit/34be4dbf87fc
which left the int problem for future adventures.
https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19
Have a nice day :)

That's just a trivially fixed bug, right? I don't recall
ever seeing a bug report or a patch for it in the past. Would
you like to send the patch, or should I?

Arnd

[Arnd Bergmann]

Post by Arnd Bergmann

Post by Thomas Schmitt
https://github.com/torvalds/linux/blob/master/fs/isofs/isofs.h line 109
https://github.com/torvalds/linux/blob/master/fs/isofs/util.c line 19
Have a nice day :)

That's just a trivially fixed bug, right? I don't recall
ever seeing a bug report or a patch for it in the past. Would
you like to send the patch, or should I?

I see you actually put the patch into the debian bug
tracker, though I can't find it on lore.kernel.org. Where
did you submit it?

Arnd

[Thomas Schmitt]

Hi,

Post by Arnd Bergmann
That's just a trivially fixed bug, right?

Yes. Very simple. Just s/int/time64_t/ .

Post by Arnd Bergmann
I don't recall ever seeing a bug report or a patch for it in the past.

I did not submit it to LKML or linux-scsi, because i got no reply to
earlier attempts to bring in my userland knowledge about ISO 9660 and
optical drives:

https://lore.kernel.org/linux-scsi/20201120140633.1673-1-***@gmx.net/T/
[PATCH] isofs: fix Oops with zisofs and large PAGE_SIZE

https://lore.kernel.org/linux-scsi/20201006094026.1730-1-***@gmx.net/T/
[PATCH v2 0/2] Fix automatic CD tray loading for data reading via sr
[PATCH v2 1/2] cdrom: delegate automatic CD tray loading to callers of
cdrom_open()
[PATCH v2 2/2] sr: fix automatic tray loading for data reading

I made more bug fixes and a wishlist patch two years ago.
But keeping them up to date with the agile kernel development is quite a
big task for me. (As said: userlander.)

Especially fs/isofs has no maintainer, so i could only submit to linux-scsi
because of the proximity to cdrom and sr. I had hoped that above two
patches would be considered as modest self-introduction, but obviously my
social skills are not sufficient.

kernelnewbies could not help me either. I am still subscribed there in
the hope to find allies. At one occasion i saw a chance when Adverg
Ebashinskii asked for low hanging fruit in fs:
https://www.mail-archive.com/search?q=Kernel+bug+tracker&l=kernelnewbies%40kernelnewbies.org
I proposed the Y2038 patch:
https://www.mail-archive.com/***@kernelnewbies.org/msg21629.html
with the offer to reduce my "Signed-off-by:" to "Suggested-by:".

This is what i think needed mending in kernel 5.10 two years ago:

[PATCH 0/3] Fix the old CD read-ahead bug for media with a single TAO track
(Most drives report the 2 unreadable TAO run-out blocks as
part of the medium capacity.)

[PATCH 0/1] sr: verify that last_written block is readable before deriving
size from it
(Size assessment of optical media in Linux is quite a mess
and can overshoot beyond the TAO run-out problem.)

[PATCH 0/4] Attribute size 0 to sr device if no readable medium is loaded
(Linux reports the size of the last loaded medium, or 2048
bytes if a blank medium is inserted.)

[PATCH 0/4] Make mount -t iso9660 -o session=N work on DVD and BD media up
to N=168
(While -o sbsector= works for all media types, session= works
only with CD media.)

[PATCH 0/1] isofs: truncate oversized Rock Ridge names to 255 bytes
(Truncation currently happens if name length is >= 254 and
then cuts off much more of the name than needed.)

When those bugs would be fixed (or mitigated in case of TAO), i hoped to
get a favor for my own hobby:

[PATCH 0/3] Introduce a new ioctl CDROM_SIMUL_CHANGE for burn programs
(Currently Linux knows about new content created via ioctl
SG_IO only after eject and reload of the Medium.)

The housekeeping aspects of kernel development are really hard for me to
master. I don't strive to become a regular contributor. But seeing those
bugs since years causes me to mention them when there is hope to meet
kernel developers.

So:
Thanks a lot for replying. Is there a chance to get you interested in the
other bugs above and maybe even my wish for ioctl CDROM_SIMUL_CHANGE ?


Have a nice day :)

Thomas

[Arnd Bergmann]

Post by Thomas Schmitt
[PATCH] isofs: fix Oops with zisofs and large PAGE_SIZE
[PATCH v2 0/2] Fix automatic CD tray loading for data reading via sr
[PATCH v2 1/2] cdrom: delegate automatic CD tray loading to callers of
cdrom_open()
[PATCH v2 2/2] sr: fix automatic tray loading for data reading
I made more bug fixes and a wishlist patch two years ago.
But keeping them up to date with the agile kernel development is quite a
big task for me. (As said: userlander.)

These look like you did everything right, and they should have been
picked up by the scsi maintainers. If you ever re-send them, I would
suggest putting the maintainers in 'To:' for the email and the mailing
list in Cc, and asking them specifically to pick up the patches.

Post by Thomas Schmitt
Especially fs/isofs has no maintainer, so i could only submit to linux-scsi
because of the proximity to cdrom and sr. I had hoped that above two
patches would be considered as modest self-introduction, but obviously my
social skills are not sufficient.

I think the hard part here is knowing who to send the patches to.
Unmaintained file systems are particularly tricky, in this case I
would have used

To: Alexander Viro <***@zeniv.linux.org.uk>
To: Jan Kara <***@suse.cz>
To: Andrew Morton <***@linux-foundation.org>
Cc: Arnd Bergmann <***@arndb.de>
Cc: Deepa Dinamani <***@gmail.com>
Cc: linux-***@vger.kernel.org
Cc: linux-***@vger.kernel.org
Cc: ***@lists.linaro.org

Can you rebase the patch on top of v6.1-rc1 and send it to
this list of people? I'll reply with a 'Reviewed-by' tag,
and one of the above should be able to pick it up.

Post by Thomas Schmitt
[PATCH 0/3] Fix the old CD read-ahead bug for media with a single TAO track
(Most drives report the 2 unreadable TAO run-out blocks as
part of the medium capacity.)
[PATCH 0/1] sr: verify that last_written block is readable before deriving
size from it
(Size assessment of optical media in Linux is quite a mess
and can overshoot beyond the TAO run-out problem.)
[PATCH 0/4] Attribute size 0 to sr device if no readable medium is loaded
(Linux reports the size of the last loaded medium, or 2048
bytes if a blank medium is inserted.)
[PATCH 0/4] Make mount -t iso9660 -o session=N work on DVD and BD media up
to N=168
(While -o sbsector= works for all media types, session= works
only with CD media.)
[PATCH 0/1] isofs: truncate oversized Rock Ridge names to 255 bytes
(Truncation currently happens if name length is >= 254 and
then cuts off much more of the name than needed.)
When those bugs would be fixed (or mitigated in case of TAO), i hoped to
[PATCH 0/3] Introduce a new ioctl CDROM_SIMUL_CHANGE for burn programs
(Currently Linux knows about new content created via ioctl
SG_IO only after eject and reload of the Medium.)
The housekeeping aspects of kernel development are really hard for me to
master. I don't strive to become a regular contributor. But seeing those
bugs since years causes me to mention them when there is hope to meet
kernel developers.
Thanks a lot for replying. Is there a chance to get you interested in the
other bugs above and maybe even my wish for ioctl CDROM_SIMUL_CHANGE ?

I mainly care about the y2038 issue here, haven't done anything interesting
on cdrom drivers in 20 years ;-)

Arnd

[Thomas Schmitt]

Hi,

Post by Arnd Bergmann
These look like you did everything right, and they should have been
picked up by the scsi maintainers.

I thought the same, but also that insisting, screaming, or trampling
won't help.

Post by Arnd Bergmann
If you ever re-send them, I would
suggest putting the maintainers in 'To:' for the email and the mailing
list in Cc,

I remember to have looked into the MAINTAINERS file and the git log for
names.
(In the case of the powerpc64 Oops, it looks like i added you in Cc:.
If i only could remember why ...)

Post by Arnd Bergmann
I think the hard part here is knowing who to send the patches to.
Unmaintained file systems are particularly tricky, in this case I
would have used
Can you rebase the patch on top of v6.1-rc1 and send it to
this list of people?

I know the word "rebase" but cannot promise that i can fill it with
substance soon ...

What kernel branches should i choose for sr and for isofs ?

Post by Arnd Bergmann
I mainly care about the y2038 issue here,

If you want to do us both a favor then bring the changes from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
into the kernel.
Feel free to take my reasoning and demonstration text. The code change is
trivial. So if at all, i would be fully recognized by "Suggested-by:".

It might be worth to verify my claims:

The only callers of iso_date() are in isofs/inode.c and isofs/rock.c
and put the result into struct inode.i_{a,c,m}time.tv_sec which is
of type time64_t.
The time value of iso_date() essentially stems from mktime64().

and to exercise the demonstration by a xorriso-made ISO.


Have a nice day :)

Thomas

[Arnd Bergmann]

Post by Thomas Schmitt

Post by Arnd Bergmann
I think the hard part here is knowing who to send the patches to.
Unmaintained file systems are particularly tricky, in this case I
would have used

This would be the correct list for the cdrom driver patches,
my list above would be for the isofs time64 patch.

Post by Thomas Schmitt

Post by Arnd Bergmann
Can you rebase the patch on top of v6.1-rc1 and send it to
this list of people?

I know the word "rebase" but cannot promise that i can fill it with
substance soon ...
What kernel branches should i choose for sr and for isofs ?

It's generally ok to just use the latest -rc1 (right now 6.1-rc1)
as the base, unless a maintainer asks you to use their tree
as a base.

Post by Thomas Schmitt

Post by Arnd Bergmann
I mainly care about the y2038 issue here,

If you want to do us both a favor then bring the changes from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
into the kernel.
Feel free to take my reasoning and demonstration text. The code change is
trivial.

Done now. I've left you as author and first Signed-off-by
though. While the change itself is trivial, the important bit
is identifying the problem, and you did that.

Post by Thomas Schmitt
The only callers of iso_date() are in isofs/inode.c and isofs/rock.c
and put the result into struct inode.i_{a,c,m}time.tv_sec which is
of type time64_t.
The time value of iso_date() essentially stems from mktime64().
and to exercise the demonstration by a xorriso-made ISO.

I could immediately tell that your patch is correct when I saw it.

Arnd

[Thomas Schmitt]

Hi,

Post by Arnd Bergmann
This would be the correct list for the cdrom driver patches,
my list above would be for the isofs time64 patch.

Good to know. I'll keep both lists in my cheat sheet.
(I guess that ***@lists.linaro.org is not appropriate for non-Y2038
patches.)

Post by Arnd Bergmann
It's generally ok to just use the latest -rc1 (right now 6.1-rc1)

Understood. (Two minutes after i sent my latest mail i got yours.)

Post by Arnd Bergmann

Post by Thomas Schmitt
If you want to do us both a favor then bring the changes from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800627#38
into the kernel.

Done now.

Thank you.

Also thanks to the bystanders for their tolerance.
At least my hijacking of this thread was a successful attempt at Y2038.


Have a nice day :)

Thomas

[Thomas Schmitt]

Hi,

Post by Thomas Schmitt

Post by Arnd Bergmann
Can you rebase the patch on top of v6.1-rc1

What kernel branches should i choose for sr and for isofs ?

Ok, that question was probably stupid.

May i understand your advise as
"on top of the newest vX.Y-rcZ in github.com/torvalds/linux"
?
(Given that my rebase attempts will not happen quite now.)


Have a nice day :)

Thomas

[Karoly Balogh]

Post by John Paul Adrian Glaubitz

Given that I develop rebootstrap, attempting to use it for a time64
bootstrap seemed quite natural. I've been talking to this with Steve
multiple times including DC22. The question was how to plug it in. In
the end I went for Arnd's suggestion to set DPKG_*_APPEND variables to
modify dpkg-buildflags. Not every package uses these flags, but a
majority do. For a survey, this is probably good enough.

I would love to do that for m68k as well. We could use this opportunity to
rebuild the m68k port with 32-bit alignment which would solve quite a number
of problems since many projects like LLVM and Qt assume a minimum alignment
of 32 bits while m68k still defaults to 16 bits.

Are you talking about time64 related alignment here, or in general?

Isn't the alignment is dictated by the 68k SysV ABI? Does this mean that
Debian and another X 68k distro (say, embedded builds), that decides not
to go ahead with this change not to be ABI compatible?

Doesn't this need compiler changes as well? (Example: natural alignment of
the stack on m68k (non-CF) is still two, enforced by the CPU behavior, and
needs changes to stackframe management if you want to change this to 4).

Charlie

[Richard]

Post by John Paul Adrian Glaubitz
I would love to do that for m68k as well. We could use this opportunity to
rebuild the m68k port with 32-bit alignment which would solve quite a number
of problems since many projects like LLVM and Qt assume a minimum alignment
of 32 bits while m68k still defaults to 16 bits.

Should have probably done that long ago when the TLS thing broke the ABI.

But.. did anyone do something similar before? I am wondering how many programs may store data on disk in binary form without any consideration for CPU alignment these days? Also, would we change the kernel and interfaces at the same time? How about boot programs and similar?

Richard