KernelTrap - Linux news, FreeBSD news, OpenBSD news, NetBSD news

In the continuining discussion about how GCC treats the volatile keyword, Linus Torvalds noted, "I just have a strong suspicion that 'volatile' performance is so low down the list of any C compiler persons interest, that it's never going to happen. And quite frankly, I cannot blame the gcc guys for it." He went on to explain, "that's especially as 'volatile' really isn't a very good feature of the C language, and is likely to get *less* interesting rather than more (as user space starts to be more and more threaded, 'volatile' gets less and less useful." "So I wouldn't expect 'volatile' to ever really generate better code. It might happen as a side effect of other improvements (eg, I might hope that the SSA work would eventually lead to gcc having a much better defined model of valid optimizations, and maybe better code generation for volatile accesses fall out cleanly out of that), but in the end, it's such an ugly special case in C, and so seldom used, that I wouldn't depend on it. "Quite frankly, I'd like there to be more competition in the open source compiler game, and that might cause some upheavals, but on the whole, gcc actually does a pretty damn good job." read more | Sponsor KernelTrap.org

"The elections for five of the ten members of the Linux Foundation Technical Advisory Board[TAB] are held every year, currently the election will be at the 2007 Kernel Summit in a BOF session," James Bottomley, the TAB chair, announced on the Linux Kernel mailing list. He noted that this voting session would be held on the evening of September 5'th or 6'th, providing an email address for sending nominations and adding that anyone is eligible, "only people invited to the kernel summit will be there in person (and therefore able to vote), but if you cannot attend, your nomination email will be read out before the voting begins." James went on to explain: "It's really just a represent the community type of role. The LF uses the TAB to get a sense of the community for various things they and their members are thinking. Conversely, the TAB was initially formed to get a set of specific objectives out of the then OSDL (Doc Fellowship, Travel Fund, NDA programme and HW lending library plus a few other things). The TAB takes proposals from the community for things it needs that require an organisation to sort out (a good example of this is the currently being acted on PCI sig membership, which will give us access to the PCI specs plus a vendor ID that the virtualisation people asked for to help with virtual device recognition)." read more | rsync.net - Secure Offsite Data

Ingo Molnar announced version 20 of his Completely Fair Scheduler patchset, offering further cleanups for the new scheduler code that will be part of the upcoming 2.6.23 kernel, "there have been lots of small regression fixes, speedups, debug enhancements and tidy-ups - many of which can be user-visible." Ingo went on to summarize: "There are nearly 100 changes - they do add up to a significant total linecount change. There was no crash bug or hang bug found in the CFS code since v19 was released. (in fact the last crash/hang bug in CFS was found and fixed in v7, more than 3 months ago, and even that crash only happened in an uncommon sw-suspend setup, not during normal use. So CFS has turned out to be a pretty robust codebase.) Nevertheless, if you had any problems (performance or behavioral) with v19 it's worth checking v20 out - and if v19 worked great for you it's worth checking out that v20 still works great =B-)" read more | rsync.net - Secure Offsite Data

A recent bug report led to a discussion about potentially dropping support for pre-4.0 versions of GCC. Adrian Bunk noted, "currently we support 6 different stable gcc release series, and it might be the right time to consider dropping support for the older ones. Are there any architectures still requiring a gcc " Russell King noted that on some architectures GCC 3.x is still preferable to the newer 4.x branch, "I want to keep support for gcc 3.4.3 for ARM for the foreseeable future. From my point of view, gcc 4 compilers have been something of a development thing as far as the ARM architecture goes. Also, gcc 3.4.3 is faster and significantly less noisy than gcc 4." When it was asked how many kernel developers use older version of GCC, Linus Torvalds explained that it really doesn't matter, "it's NOT about 'kernel developers'. It's about random people testing kernels. If we make it harder for people to test kernels, we're going to lose. So no, I vote for *not* cutting off old gcc versions unless it's absolutely fatal." read more | Sponsor KernelTrap.org

Jungseung Lee announced the first public release of gitstat, "a GPL'd, web-based git statistics/monitoring system." He explains, "it retrieves a specified git tree, analyzes changesets, and shows graphical information like the number of changesets per day, the number of people who submitted changesets for a specific version(tag), etc." The link above offers a graphical view of Linus' mainline 2.6 kernel tree, with daily commit statistics, monthly commit statistics, kernel release frequency, and per-author statistics. Jungseung noted: "Gitstat was derived from kfm (kernel feature monitor) which was originally developed by Keun-Sik Lim and Sang-Bae Lee of Samsung Electronics and currently maintained and developed by Jeong-Seung Lee and Soon-Son Kwon(Shawn) of Samsung Electronics. Kfm was inspired from Jon Corbet of lwn.net when he analyzed the git tree and Greg KH when he presented similar status report at OLS2007. We thought it would be interesting information every day." read more | rsync.net - Secure Offsite Data

"People who think 'volatile' really matters are just fooling themselves," Linus Torvalds quipped during a lengthy discussion on the Linux Kernel mailing list. The thread began with a series of patches to "make atomic_read() behave consistently across all architectures" which included "removing the volatile keyword from all atomic_t and atomic64_t definitions that currently have it, and instead explicitly [casting] the variable as volatile in atomic_read()." Earlier in the discussion Linus had suggested that while it didn't actually fix any bugs it did help hide bugs and make them less likely to be triggered, "and hey, sometimes 'hiding bugs well enough' is ok. In this case, I'd argue that we've successfully *not* had the volatile there for eight months on x86-64, and that should tell people something. " But later in the discussion he related it to superstitions like the fear of a black cat crossing the road, which "has no bigger and longer-lasting direct affects": "In other words, this whole discussion has just convinced me that we should *not* add back 'volatile' to 'atomic_read()' - I was willing to do it for practical and 'hide the bugs' reasons, but having seen people argue for it, thinking that it actually fixes something, I'm now convinced that the *last* thing we should do is to encourage that kind of superstitious thinking." read more | rsync.net - Offsite Unix Backups

In a series of 5 patches, Jesper Juhl propsed moving 4K stacks from a debug feature to a non-debug feature, defaulting it to be enabled in the -mm tree. He referred back to a lengthy earlier discussion in which he had proposed making 4K stacks the default in the mainline kernel, then added: "Based on the comments in that thread I conclude that 4KSTACKS are not really considered a debug-only feature any longer, but the time is not right (yet) to make them the default - and it's certainly not yet the time to get rid of 8K stacks." "In that thread I promised to provide some patches that would lift 4KSTACKS out of debug-only feature status, which is what the first two patches in this series do. I also said I would provide a patch to make 4KSTACKS 'default y' to get more testing, but restrict that patch to -mm - that's the fifth patch in this series. Patches 3 & 4 in this series move the config option out of the Kernel hacking menu and into Processor types and features". read more | rsync.net - Offsite Unix Backups

In a June of 1992 posting to the linux-activists mailing list, Linus Torvalds described the original Linux scheduler noting, "the scheduler in linux is pretty simple, but does a reasonably good job at giving good IO response while not being too unfair against cpu-bound processes." A year later, Linus posted a more detailed description of the scheduler noting, "the linux scheduling algorithm is one of the simplest ones possible". Comments in the original 254 line sched.c file read, "'schedule()' is the scheduler function. This is GOOD CODE! There probably won't be any reason to change this, as it should work well in all circumstances (ie gives IO-bound processes good response etc). The one thing you might take a look at is the signal-handler code here." Comments in the current 6,709 line sched.c file show the first changes being made in 1996 by Dave Grothe, "to fix bugs in semaphores and make semaphores SMP safe". Two years later Andrea Arcangeli is credited with implementing "schedule_timeout() and related stuff". It was not until 2002, ten years after Linus' original code was written, that the scheduler received a complete rewrite, "new ultra-scalable O(1) scheduler by Ingo Molnar: hybrid priority-list and round-robin design with an array-switch method of distributing timeslices and per-CPU runqueues." Con Kolivas is credited with "interactivity tuning" in 2003, and Nick Piggin added "scheduler domains" in 2004. A more recent rewrite of the scheduler happened in April, again by Ingo Molnar, this time with his Completely Fair Scheduler. read more | Sponsor KernelTrap.org

In a June of 1992 posting to the linux-activists mailing list, Linus Torvalds described the original Linux scheduler noting, "the scheduler in linux is pretty simple, but does a reasonably good job at giving good IO response while not being too unfair against cpu-bound processes." A year later, Linus posted a more detailed description of the scheduler noting, "the linux scheduling algorithm is one of the simplest ones possible". Comments in the original 254 line sched.c file read, "'schedule()' is the scheduler function. This is GOOD CODE! There probably won't be any reason to change this, as it should work well in all circumstances (ie gives IO-bound processes good response etc). The one thing you might take a look at is the signal-handler code here." Comments in the current 6,709 line sched.c file show the first changes being made in 1996 by Dave Grothe, "to fix bugs in semaphores and make semaphores SMP safe". Two years later Andrea Arcangeli is credited with implementing "schedule_timeout() and related stuff". It was not until 2002, ten years after Linus' original code was written, that the scheduler received a complete rewrite, "new ultra-scalable O(1) scheduler by Ingo Molnar: hybrid priority-list and round-robin design with an array-switch method of distributing timeslices and per-CPU runqueues." Con Kolivas is credited with "interactivity tuning" in 2003, and Nick Piggin added "scheduler domains" in 2004. A more recent rewrite of the scheduler happened in April, again by Ingo Molnar, this time with his Completely Fair Scheduler. read more | Sponsor KernelTrap.org

In an overwhelmingly large series of 556 patches, Joe Perches attempted to track down maintainers for a significant number of files within the Linux kernel source tree. He explained, "I grew weary of looking up the appropriate maintainer email address(es) to CC: for a patch", adding a new line format to the kernel MAINTAINERS file parsed by a new get_maintainer.pl script. Much of the feedback was criticism of the large number of patches that flooded the inboxes of all subscribers to the Linux Kernel Mailing List. Others suggested that the information would be better extracted from Git than from source files. When Joe asked for better ideas for achieving his end goal, Alan Cox suggested, "working off the git tree as it shows who actually is making changes/updating stuff recently and why which is a major clue when tracing bugs". Chris Wright pointed out, "I think this data will easily become stale. What is the point again?" going on to add "between git (or gitweb), existing MAINTAINERS and a bit of common sense (or extra sleuthing), I never perceived a significant problem." Adrian Bunk countered, "for active kernel developers like you and me it's not a problem. But for other people it's non-trivial to always figure out who the maintainer of some part of the kernel is." read more | rsync.net - Secure Offsite Data

"Either people really are calming down, and figuring out that we're in the stabilization phase," Linus Torvalds began in announcing 2.6.23-rc3, "or it's just that it's the middle of August, and most everybody at least in Europe are off on vacation." The actual source-level changes can be browsed via the kernel.org gitweb interface. Linus went on to summarize: "Regardless of why, -rc3 is out, and doesn't have the tons of changes that -rc2 did. But there's some scheduler updates, sparc64 and powerpc changes, and random driver updates (the lpfc SCSI driver kind of stands out in the diffstat). Shortlog appended, I don't know what I can add to it.. Please do give it a good testing, unless you're on a beach sunning yourself (and who are we kidding: you're pasty white, and sand is hard to get out of the keyboard - beaches are overrated)." read more | Sponsor KernelTrap.org

Greg KH and Chris Wright have been maintaining a -stable 2.6.x.y patchset for the 2.6.x and 2.6.(x-1) kernels since March of 2005. Thus, with the current stable release being 2.6.22, they maintain -stable patches for 2.6.22 and 2.6.21. 2.4 stable kernel maintainer Willy Tarreau noted the currently high patch rate in each of the 2.6 -stable trees and decided to maintain -stable patches against the 2.6.20 tree until things calm down. Adrian Bunk also continues to maintain a -stable 2.6.16 branch of the Linux kernel. Willy explained about his new 2.6.20 -stable patches: "I proposed Chris and Greg to continue issuing a few more 2.6.20 releases during the time needed for 2.6.21 and 2.6.22 to show a significant drop in their patch rates, which hopefully will be just a matter of a few releases. "My goal is *not* to do all the hard work they do, but just to backport from their patches those which are meaningful for 2.6.20. For this reason, 2.6.20 releases will always be slightly late and should not contain patches not merged in more recent releases." read more | rsync.net - Secure Offsite Data

"BACK UP ANY IMPORTANT DATA," began the Linux 0.10 installation instructions. "Linux accesses your hardware directly, and if your hardware differs from mine, you could be in for a nasty surprise. Doublecheck that your hardware is compatible: AT style harddisk, VGA controller." The installation guide explained that there were five major steps in getting Linux installed and running on your computer, including the above first step of backing up the system. The second step was to use Minix and the mkfs command to create a new filesystem on an empty partition of your hard drive. Third you used dd to write the 'boot' and 'root' Linux disk images to floppy disks. The fourth step was actually booting from the floppies, "having a floppy as root-device isn't very fast (especially on a machine with less than 6MB total ram -> small buffer cache), but it works (I hope)." The final step was mounting the empty hard disk partition, copying the files from the floppy disks to the partition, and creating the necessary /dev files with mknod, "you should now have a filesystem you [can] boot from. Play around a bit, try to get acquainted with the new system. Log out when you've had enough." The document noted that while it was possible to install Linux using DOS, the instructions were intended for people using Minix: "In general, this version is still meant for people with minix: they are more used to the system, and can do some things that DOS-based persons cannot. If you have only DOS, expect some troubles. As the version number suggests, this is still not the final product." read more | rsync.net - Offsite Unix Backups

Ingo Molnar pushed a series of patches to his Completely Fair Scheduler code upstream that were merged into the mainline kernel. He explained the reason for so many small patches, "the main reason is the safe and gradual elimination of a widely used 64-bit function argument: the 64-bit 'now' timestamp." In addition to the many small patches removing the 64-bit "now" timestamp, he noted: "These changes are necessary for 3 reasons: firstly they address the 'there's too much 64-bit stuff in the scheduler' observation. Secondly, it's not directly visible but these changes also act as a correctness fix for an obscure (and minor) but not-too-pretty-to-fix accounting bug: idle_balance() had its own internal notion of 'now', separate from that of schedule(). Thirdly, this debloats sched.o quite significantly." read more | Sponsor KernelTrap.org

Following a recent merge request, Linus Torvalds stressed that he was serious about not wanting to merge any big changes after the merge window closes, "get the changes in before -rc1, or just *wait*. If they aren't ready before the merge window opens, they simply shouldn't be merged at all." Jeff Garzik reiterated, "once -rc1 is out there, that means the focus should be on stabilizing the existing codebase. Pushing a big driver update means that effort must restart from scratch. We just don't want to go down that road, which a big reason for the merge window in general." Further when it was noted that the recent changes were heavily tested by the vendor, Jeff stressed the importance of community testing: "Take a lesson from when I was on Linus's shit-list... twice: Twice, Intel submitted an e1000 update after the merge window closed. Twice, they claimed the driver passed their quite-exhaustive internal testing. And twice, the most popular network driver broke for large masses of users because I took a hardware vendor's word on testing rather than rely on the testing PROVEN to flush out problems: public linux kernel testing. "I'm not singling out Intel, there are plenty of other hardware vendors that repeat the exact same pattern." read more | rsync.net - Offsite Unix Backups

In a recent lkml thread, Linus Torvalds was involved in a discussion about mounting filesystems with the noatime option for better performance, "'noatime,data=writeback' will quite likely be *quite* noticeable (with different effects for different loads), but almost nobody actually runs that way." He noted that he set O_NOATIME when writing git, "and it was an absolutely huge time-saver for the case of not having 'noatime' in the mount options. Certainly more than your estimated 10% under some loads." The discussion then looked at using the relatime mount option to improve the situation, "relative atime only updates the atime if the previous atime is older than the mtime or ctime. Like noatime, but useful for applications like mutt that need to know when a file has been read since it was last modified." Ingo Molnar stressed the significance of fixing this performance issue, "I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_." He submitted some patches to improve relatime, and noted about atime: "It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" read more | rsync.net - Secure Offsite Data

Some entertaining lguest documentation discussed in an earlier story was merged into the mainline kernel with the commit message, "the netfilter code had very good documentation: the Netfilter Hacking HOWTO. Noone ever read it. So this time I'm trying something different, using a bit of Knuthiness." Both Netfliter and lguest, as well as the documentation for both, were written by Rusty Russell. He describes the lguest driver as, "a simple hypervisor for Linux on Linux. Unlike kvm it doesn't need VT/SVM hardware. Unlike Xen it's simply 'modprobe and go'. Unlike both, it's 5000 lines and self-contained." Downloading the 2.6.23-rc2 kernel and looking in the "drivers/lguest/" directory I found a simple README that kicks off an interesting documentation process, beginning, "welcome, friend reader, to lguest." It goes on to note, "I can't think of many 5000-line projects which offer both such capability and glimpses of future potential; it is an exciting time to be delving into the source!" At the end of the included README is a hint as to how to find the rest of the documentation, which is embedded inline within all the lguest files. Read on to begin the exploration into lguest and its documentation. read more | Sponsor KernelTrap.org

Nick Piggin used 'git bisect' to track a lmbench regression to the main CFS commit, leading to an interesting discussion between Nick and Ingo Molnar. Ultimately the regression was tracked down to the temporary configurability of the scheduler while it is tuned for optimal performance, "one reason for the extra overhead is the current tunability of CFS, but that is not fundamental, it's caused by the many knobs that CFS has at the moment." The solution, already coded but not yet merged in the mainline kernel "changes those knobs to constants, allowing the compiler to optimize the math better and reduce code size," and as a result result, "CFS can be faster at micro-context-switching than 2.6.22." Ingo described the lmbench configuration in question as a "micro-benchmark", and noted that with a macro-benchmark better performance was more pronounced, "because with CFS the _quality_ of scheduling decisions has increased. So even if we had increased micro-costs (which we wont have once the current tuning period is over and we cast the CFS parameters into constants), the quality of macro-scheduling can offset that, and not only on the desktop!" He summarized, "that's why our main focus in CFS was on the macro-properties of scheduling _first_, and then the micro-properties are adjusted to the macro-constraints as a second layer." read more | Sponsor KernelTrap.org

"So I tried to hold people to the merge window," Linus Torvalds began in announcing the 2.6.23-rc2 kernel, "and said no to a few pull requests, but this whole '-rc2 is the new -rc1' thing is a disease, and not only is -rc2 late, it's bigger than it should be. Oh, well." He noted that over 250 people contributed patches between -rc1 and -rc2, adding: "A lot of the changes are small, and a lot of them really are fixes, but there's a MIPS merge in there too, and some absolutely _huge_ diffs due to some drivers undergoing Lindent cleanups (28 _thousand_ lines changes in advansys.c, and the PNP files got Lindented too, although those weren't nearly as big). "But if you ignore the Lindent changes, the MIPS merge, the lguest documentation updates, and the MPT fusion driver changes, and the removal of the broken arm26 support, the rest of the changes really aren't that big." read more | Sponsor KernelTrap.org

KernelTrap is actively seeking sponsorship for September and October of 2007. read more | Sponsor KernelTrap.org