LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 02-07-2023, 09:42 AM   #1696
elcore
Senior Member
 
Registered: Sep 2014
Distribution: Slackware
Posts: 1,754

Rep: Reputation: Disabled

Quote:
Originally Posted by Petri Kaukasoina View Post
"That is saving bandwidth for you" is not quite correct. The Slackware txz packages are compressed tar files, and recompressing already compressed firmware data is not efficient bandwidthwise. Didier did not tell how the compressed package sizes compare, but the txz of compressed firmware would be larger than "the genuine Slackware package".
Probably didn't have to state the obvious.
Stronger compression reduces file size, which is why for example youtube requires a lot of CPU power to encode/decode and less bandwidth to actually use.
And whatever, I'm not trying to argue, just saying unpacking "ucode" and few other files with mc, is multiple times faster than doing "upgradepkg".
Same thing goes for kernel-source because of many checks upgradepkg does on the huge directory tree and great number of files. I don't think it's xz problem, TBH.
 
Old 02-07-2023, 10:40 AM   #1697
Didier Spaier
LQ Addict
 
Registered: Nov 2008
Location: Paris, France
Distribution: Slint64-15.0
Posts: 11,077

Rep: Reputation: Disabled
Quote:
Originally Posted by Petri Kaukasoina View Post
"That is saving bandwidth for you" is not quite correct. The Slackware txz packages are compressed tar files, and recompressing already compressed firmware data is not efficient bandwidthwise. Didier did not tell how the compressed package sizes compare, but the txz of compressed firmware would be larger than "the genuine Slackware package".
Correct. In Slackware 15.0:
Code:
PACKAGE NAME:     kernel-firmware-20220531_eaee2da-noarch-1slint
COMPRESSED PACKAGE SIZE:     313M
UNCOMPRESSED PACKAGE SIZE:     333M

PACKAGE NAME:     kernel-firmware-20220124_eb8ea1b-noarch-1
COMPRESSED PACKAGE SIZE:     196M
UNCOMPRESSED PACKAGE SIZE:     792M
So no, this does not save bandwidth for me. More exactly for Tadgy, who kindly hosts the Slint repositories for free, thanks Darren!

As an aside, the size on Slint 15.0 is slightly smaller than in Slackware15.0, due to a different file system used.
 
Old 02-07-2023, 10:48 AM   #1698
garpu
Senior Member
 
Registered: Oct 2009
Distribution: Slackware
Posts: 1,610

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Be interesting to get Hazel to weigh in. I know she's got some bandwith issues at her place, so this is something that's in her area of expertise.
 
Old 02-07-2023, 10:48 AM   #1699
Didier Spaier
LQ Addict
 
Registered: Nov 2008
Location: Paris, France
Distribution: Slint64-15.0
Posts: 11,077

Rep: Reputation: Disabled
Quote:
Originally Posted by elcore View Post
Same thing goes for kernel-source because of many checks upgradepkg does on the huge directory tree and great number of files. I don't think it's xz problem, TBH.
No, it's rather a bash problem. spkg is way faster in this case because it is a compiled application written in C.
 
Old 02-07-2023, 10:51 AM   #1700
elcore
Senior Member
 
Registered: Sep 2014
Distribution: Slackware
Posts: 1,754

Rep: Reputation: Disabled
Quote:
Originally Posted by Didier Spaier View Post
Correct. In Slackware 15.0:
Code:
PACKAGE NAME:     kernel-firmware-20220531_eaee2da-noarch-1slint
COMPRESSED PACKAGE SIZE:     313M
UNCOMPRESSED PACKAGE SIZE:     333M

PACKAGE NAME:     kernel-firmware-20220124_eb8ea1b-noarch-1
COMPRESSED PACKAGE SIZE:     196M
UNCOMPRESSED PACKAGE SIZE:     792M
So no, this does not save bandwidth for me..
So I guess trade some minor reduction of install-time requirements, for increased traffic and increased run-time requirements?
How is that rational? I thought we're talking about splitting firmware, in order to reduce traffic and runtime-requirements.
This took some unexpecting turn, I didn't realize at first what you were trying to achieve.
 
Old 02-07-2023, 10:59 AM   #1701
EdGr
Senior Member
 
Registered: Dec 2010
Location: California, USA
Distribution: I run my own OS
Posts: 1,003

Rep: Reputation: 474Reputation: 474Reputation: 474Reputation: 474Reputation: 474
Quote:
Originally Posted by Pigi_102 View Post
I would like to have a set of smaller firmware kernel packages instead of a big ( huge ?) one.
Would it be possibile or is an hard thing to realize ?
You can easily remove the three worst offenders.

Code:
% cd /lib/firmware
% du -s * | sort -n | tail -n 5
50840	mrvl
65040	amdgpu
71292	mellanox
92728	qcom
143580	netronome
Ed
 
Old 02-07-2023, 11:04 AM   #1702
Didier Spaier
LQ Addict
 
Registered: Nov 2008
Location: Paris, France
Distribution: Slint64-15.0
Posts: 11,077

Rep: Reputation: Disabled
Quote:
Originally Posted by elcore View Post
So I guess trade some minor reduction of install-time requirements, for increased traffic and increased run-time requirements?
increased traffic, yes, run-time requirements, no, as Petri has clearly explained.Did you really measure the time needed to load the few firmware files you are using?
Quote:
This took some unexpected turn, I didn't realize at first what you were trying to achieve.
I achieve two things:
  1. Less space on disk needed (around 460G saved). This is useful also for Slackware.
  2. Specifically for Slint allow to ship the whole kernel-firmware package in the initramfs of the installer, to allow using any wifi card for net install, while not requiring more than 2G of RAM and not wasting my time maintaining split firmware packages.
As an aside I also ship zstd compressed kernel modules in the initramfs.

Last edited by Didier Spaier; 02-07-2023 at 12:47 PM. Reason: Typo fix.
 
3 members found this post helpful.
Old 02-07-2023, 11:10 AM   #1703
LuckyCyborg
Senior Member
 
Registered: Mar 2010
Posts: 3,605

Rep: Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470
Quote:
Originally Posted by elcore View Post
I thought we're talking about splitting firmware, in order to reduce traffic and runtime-requirements.
Honestly, I do NOT miss in this forum the traditional Ubuntu threads like "My CrapOne WiFi card does not work. Please help me!"

I for one I believe that the idea of splitting firmware in Slackware (and Ubuntu too) is worth of an Ig Nobel Prize

You are nuts, people? Even the Ubuntu, with it's apt-get and a hell of hardware auto-detection, have a hard time with those split firmware. On Slackware this will be really, but really nasty - unless, of course, in the traditional Slackware way, shall be installed all firmware packages. And we will end with installing 100 packages instead of one.

Last edited by LuckyCyborg; 02-07-2023 at 11:49 AM.
 
5 members found this post helpful.
Old 02-07-2023, 11:21 AM   #1704
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,938

Rep: Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568
Quote:
Originally Posted by Didier Spaier View Post
As an aside I also ship zstd compressed kernel modules in the initramfs.
I was just going to mention that in Slackware-11.0 the modules were gzipped but not after that. I wonder why. Man and info pages are still gzipped. There is the same trade-off there: gzipped man and info pages take less space on disk but make txz packages larger and need CPU cycles to browse on elcore's system.
 
1 members found this post helpful.
Old 02-07-2023, 11:23 AM   #1705
elcore
Senior Member
 
Registered: Sep 2014
Distribution: Slackware
Posts: 1,754

Rep: Reputation: Disabled
Quote:
Originally Posted by Didier Spaier View Post
No, it's rather a bash problem. spkg is way faster in this case because it is a compiled application written in C.
Technically, it's pkgtools logic and not bash fault, because mc also uses bash commands to unpack files from txz to filesystem.
There's room to improve this feature efficiently; my solution (for firmware and source package) is to bypass upgradepkg and just do rm-rf followed by installpkg.
Usually, not something fit for general public distribution.
But if you have ideas on how to improve it without increasing any requirements, that'd be great.
 
Old 02-07-2023, 11:48 AM   #1706
Petri Kaukasoina
Senior Member
 
Registered: Mar 2007
Posts: 1,938

Rep: Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568Reputation: 1568
Quote:
Originally Posted by elcore View Post
my solution (for firmware and source package) is to bypass upgradepkg and just do rm-rf followed by installpkg.
Usually, not something fit for general public distribution.
But if you have ideas on how to improve it without increasing any requirements, that'd be great.
Note that there are six other packages putting firmware files there besides kernel-firmware. You may have removed more than you thought.
 
3 members found this post helpful.
Old 02-07-2023, 12:01 PM   #1707
ZhaoLin1457
Senior Member
 
Registered: Jan 2018
Posts: 1,051

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Quote:
Originally Posted by LuckyCyborg View Post
Even the Ubuntu, with it's apt-get and a hell of hardware auto-detection, have a hard time with those split firmware. On Slackware this will be really, but really nasty - unless, of course, in the traditional Slackware way, shall be installed all firmware packages. And we will end with installing 100 packages instead of one.
I've always wondered how these distributions manage the firmware packages. I remember that in Fedora, these firmware packages appear as dependencies.
 
Old 02-07-2023, 12:04 PM   #1708
elcore
Senior Member
 
Registered: Sep 2014
Distribution: Slackware
Posts: 1,754

Rep: Reputation: Disabled
Quote:
Originally Posted by Petri Kaukasoina View Post
Note that there are six other packages putting firmware files there besides kernel-firmware. You may have removed more than you thought.
Maybe, on full install. I don't have/use any, apart from ucode and 2x nic.
 
Old 02-07-2023, 12:10 PM   #1709
LuckyCyborg
Senior Member
 
Registered: Mar 2010
Posts: 3,605

Rep: Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470Reputation: 3470
Quote:
Originally Posted by ZhaoLin1457 View Post
I've always wondered how these distributions manage the firmware packages. I remember that in Fedora, these firmware packages appear as dependencies.
Well, there's a system of hardware auto-detection, which tries to find the kernel modules which are needed for the system, then this is mapped to packages which contains those modules, then the associated firmware packages are dependencies of the kernel modules packages.

So, as example: when you plug a new WiFi dongle in your computer, Ubuntu asks you to install the support packages. That's great, WHEN it works. Because sometimes this hardware auto-detection logic fails, and the user ends with a WiFi dongle which does not work at all.

BUT, those are the main features used: the hardware auto-detection and the packages management with remote binary repository support - and of course, the packages dependency resolution is fundamental. And many, many packages of firmware and kernel modules.

Last edited by LuckyCyborg; 02-07-2023 at 12:15 PM.
 
1 members found this post helpful.
Old 02-07-2023, 12:27 PM   #1710
ZhaoLin1457
Senior Member
 
Registered: Jan 2018
Posts: 1,051

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Quote:
Originally Posted by LuckyCyborg View Post
Well, there's a system of hardware auto-detection, which tries to find the kernel modules which are needed for the system, then this is mapped to packages which contains those modules, then the associated firmware packages are dependencies of the kernel modules packages.

So, as example: when you plug a new WiFi dongle in your computer, Ubuntu asks you to install the support packages. That's great, WHEN it works. Because sometimes this hardware auto-detection logic fails, and the user ends with a WiFi dongle which does not work at all.

BUT, those are the main features used: the hardware auto-detection and the packages management with remote binary repository support - and of course, the packages dependency resolution is fundamental. And many, many packages of firmware and kernel modules.
I agree with you that automatically installing only the firmware required by the installer, slackpkg or pkgtools would be difficult in Slackware.

But, on the other hand, if the kernel firmware were divided into 100 packages, advanced users like is @elcore or @Hazel, for example, could install only the firmware packages they need, and the rest could end up in the slackpkg blacklist.

Last edited by ZhaoLin1457; 02-07-2023 at 12:35 PM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Apache 2.4 requests to non-SSL site with "Upgrade-Insecure-Requests: 1" and no trailing / get redirected to default site owendelong Linux - Server 2 06-22-2021 02:08 PM
[SOLVED] Requests for -current (20151216) rworkman Slackware 3441 12-28-2017 03:50 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 10:10 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration