Talk:Ext3

From Wikipedia, the free encyclopedia

[edit] The Second Extended File system

The Second Extended File system was devised (by Rémy Card) as an extensible and powerful file system for Linux. It is also the most successful file system so far in the Linux community and is the basis for all of the currently shipping Linux distributions.

The EXT2 file system, like a lot of the file systems, is built on the premise that the data held in files is kept in data blocks. These data blocks are all of the same length and, although that length can vary between different EXT2 file systems the block size of a particular EXT2 file system is set when it is created (using mke2fs ). Every file's size is rounded up to an integral number of blocks. If the block size is 1024 bytes, then a file of 1025 bytes will occupy two 1024 byte blocks. Unfortunately this means that on average you waste half a block per file. Usually in computing you trade off CPU usage for memory and disk space utilisation. In this case Linux, along with most operating systems, trades off a relatively inefficient disk usage in order to reduce the workload on the CPU. Not all of the blocks in the file system hold data, some must be used to contain the information that describes the structure of the file system. EXT2 defines the file system topology by describing each file in the system with an inode data structure. An inode describes which blocks the data within a file occupies as well as the access rights of the file, the file's modification times and the type of the file. Every file in the EXT2 file system is described by a single inode and each inode has a single unique number identifying it. The inodes for the file system are all kept together in inode tables. EXT2 directories are simply special files (themselves described by inodes) which contain pointers to the inodes of their directory entries.

Figure shows the layout of the EXT2 file system as occupying a series of blocks in a block structured device. So far as each file system is concerned, block devices are just a series of blocks which can be read and written. A file system does not need to concern itself with where on the physical media a block should be put, that is the job of the device's driver. Whenever a file system needs to read information or data from the block device containing it, it requests that its supporting device driver reads an integral number of blocks. The EXT2 file system divides the logical partition that it occupies into Block Groups. Each group duplicates information critical to the integrity of the file system as well as holding real files and directories as blocks of information and data. This duplication is neccessary should a disaster occur and the file system need recovering. The subsections describe in more detail the contents of each Block Group. —Preceding unsigned comment added by 202.83.49.25 (talk • contribs) 05:16, 4 November 2004

[edit] an xattr is a fork

An ext2 xattr very similar to an HPFS EA, which the fork page considers to be a type of fork. Linux does in fact support two OS/2 filesystems, HPFS and JFS, using the same API as is used for ext3. The only reason you couldn't store most MacOS-style resource data is a size limit; small ones would work just fine though.

24.110.60.225 02:39, 3 January 2006 (UTC)

No, they're not. Forks can be read and written like normal files (i.e. you can call read(2), write(2), ftruncate(2), etc. on them). As such, Solaris 10's xattrs (on UFS or ZFS) are forks, as are NTFS's named streams, HFS's resource and data forks, and HFS+'s forks. An xattr on Linux is basically a named extension to struct stat. They are of limited size, and can only be atomically fetched, replaced, or deleted. They are not forks. Fork's definition of what a fork is is wrong, but I don't care to deal with it. --NJM 05:44, 3 January 2006 (UTC)

A fork is any kind of user defined data attached to a file that doesn't belong to file data or metadata.

They are just name=value and don't have to be unlimited in size.

So, xattr are forks.

—Claunia 12:57, 3 January 2006 (UTC)

By that definition, Mac OS doesn't have forks. --NJM 02:01, 4 January 2006 (UTC)

It has. ICON, ICNS, MENU, CODE, BNDL, etc. You can consider them as independent forks.

Or you can consider it has a big fork named "resource".

Apart from that 10.4 added support for other forks, named as user want.

—Claunia 13:43, 4 January 2006 (UTC)

All those things you just mentioned are either file data or metadata. (Except the OS X additions, but I wasn't talking about OS X.) --NJM 03:47, 5 January 2006 (UTC)

ICON, ICNS, MENU, CODE, BNDL, so on, are resources of Macintosh resource fork, and not file data or metadata, and are considered a different fork (alternate data stream).

LONGNAME, CHARCODE, so on, are extended attributes of OS/2, and not file data or metadata, and are considered a different fork (alternate data stream).

$Macintosh_Finder_Info, $Extended_Attribute, so on, are ADSs of NTFS, and not file data or metadata, and are considered a different fork (alternate data stream).

What have xattr that make them so very different to any of these to not be considered a fork or alternate data stream?

—Claunia 08:50, 5 January 2006 (UTC)

ICON/ICNS/MENU/CODE/BNDL/etc are contained in the Macintosh resource fork and not properly forks on their own. However, Macs do support two forks per file -- the well-known data and resource forks.

The NTFS alternate data streams are forks.

OS/2 EAs aren't forks either, for the exact same reasons that Linux xattrs aren't.

The important details is that alternate forks (whether the Mac resource fork, alternate streams on NTFS, xattrs on Solaris, named attributes in NFSv4, etc.) behave exactly like the default unnamed fork that all files have (once you've jumped passed the initial hurdle of opening them) -- they have a file pointer and are seekable, you can read and write them in whole or in part, truncate them, map them into memory, lock regions for your own exclusive use, etc. (Admittedly, doing these things to a Mac resource fork takes more effort than other operating systems.)

Linux xattrs (as well as OS/2 Extended Attributes and BeOS attributes) don't behave like the default unnamed fork -- you can't open them (i.e. acquire a handle to an attribute like a normal file data stream), you can't seek them, you can't truncate existing attributes, you can't memory map them, you can't lock them, etc. The operations that can be performed on xattrs are much more limited -- you can get an existing attribute, you can set an attribute (which will either create a new attribute or atomically replace an existing attribute), and you can remove an attribute entirely. That's all. This moves xattrs into an entirely different problem space than forks.

Forks are basically a way of attaching more files to a file, while xattrs are a way of extending file metadata to new and interesting areas -- the primary use of xattrs on Linux systems are for the storage of ACLs and MAC labels. Along with the system xattr namespace (with it's kernel-enforced content restrictions), there's also the user namespace, which apps like Samba are free to use to store extra bits of information that the traditional Unix struct stat doesn't keep around (like file creation times).

xattrs don't look like forks, they don't act like forks, they aren't used to store the same kinds of things as forks, they aren't forks. --NJM 01:11, 8 January 2006 (UTC)

In computing, a fork is additional data associated with a file system object. from Fork_(filesystem).

ADS, Resource Fork, EAs, xattr, are that. —Claunia 13:21, 8 January 2006 (UTC)

As NJM and Claunia appear to disagree on the definition of a fork I would suggest either to continue this discussion on the corresponding talk page or—as a compromise—to add something like the following to this article:

While Linux xattrs represent additional data associated with a file system object and so may be called forks according to the most general definition, the operations that can be performed on xattrs on Linux are much more limited than the operations on xattrs on Solaris, Mac resource forks, alternate streams on NTFS, named attributes in NFSv4 etc. in that one cannot acquire a handle to an attribute like a normal file data stream nor can Linux xattrs be seeked etc.

That way both points of view of this discussion could be represented and the value of the article be increased. —Tobias Bergemann 22:00, 8 January 2006 (UTC)

[edit] Variable block size

The article says under "Disadvantages" that "many of the on-disk structures are similar to those of ext2. Because of that, ext3 lacks a number of features of more recent designs, such as dynamic allocation of i-nodes, variable block sizes". The Ext2 article on the other hand claims that "ext2 ... has variable length block size". Either there is a contraditcion or poor use of words.

xerces8 --86.61.44.206 21:14, 4 March 2006 (UTC)

[edit] HTree in ext2/3 the same as the linked to "hash tree"?

The htree used to represent directory entries in ext2/3 filesystems is an implementation of BTrees. It is not an implementation of the algorithm mentioned in the htree article. It is called an htree because the directory entry names (i.e. filenames and directory names) - the keys for a lookup operation - are hashed. Or this is what I understand. Do let me know what you'll think.

Veliath 07:20, 11 April 2006 (UTC)

You are correct in pointing out this error in the article. The "H-Tree" (Hash Tree, Merkle Tree) structure currently referenced is NOT the same as the HTree structure used in ext2/3. Daniel Phillips simply coined this term (see his 2001 paper at the Ottawa Linux Conference) to describe the custom structure he developed.

-- phil hutto (pwh@cc.gatech.edu 1:17 am , 25 April 2006 (EST) —Preceding unsigned comment added by 68.155.131.205 (talk • contribs) 05:18, 25 April 2006

[edit] ext2 and ext3 definitions

Further is the htree based directory indexing a new feature in ext3? It seems to be available in ext2 as well. Should it be introduced in the section on ext3's features over its predecessor? Once again do let me know what you'll think. --Veliath 06:37, 9 May 2006 (UTC)

ext2 doesn't support the htree opton. --NJM 01:56, 12 May 2006 (UTC)

So is ext3 the name for an updated ext2 with a new version number or simply an ext2 file-system with a journal? I ask because an ext2 file-system can be created without a journal and using directory indices. --Veliath 18:33, 20 May 2006 (UTC)

ext3 is the updated filesystem (and site of all new development). My understanding is that ext2 only gets new stuff when an ext3 filesystem would become unintelligible or when there's very little complexity involved. So, the htree option wasn't added to ext2 because it's transparently backward compatible (htree unaware drivers just ignore the tree block and do an ordinary linear search through the directory) and the implementation itself was somewhat complex. (On the otherhand, things like xattr (with the accompanying SELinux labels and ACL support) were added because they weren't very difficult to implement and not including them would render the fs unreadable by ext2 (or else readable but potentially insecure)). --NJM 02:57, 21 May 2006 (UTC)

The file src/linux/Documentation/filesystems/ext3.txt has a line in the intro that reads "ext3 is ext2 filesystem enhanced with journalling capabilities". It goes on to describe only journaling - hence my query. --Veliath 15:17, 22 May 2006 (UTC)

[edit] List of distros using ext3

I removed the list of distros using ext3 because I thought it was getting too long and was largely useless. There are literally hundreds of Linux disributions, many or most of which that use ext3 by default and listing all of them would be a waste of space and mostly irrelevant to the article (The fact that ext3 is primarily used by Linux / used by most Linux distros is more important than the actual list itself). Pruning the list down to a limited selection of distros would only lead to an argument over which distros should be listed, so removing the list entirely was the better choice. --NJM 17:43, 13 May 2006 (UTC)

[edit] Deletion Recovery

Look at this: Stellar Phoenix Linux - ext2/ext3 Data Recovery Software. As far as I know, there's no other application that can recover deleted files on a ext3 filesystem. On many FAQs about ext3 you can read that it's hard to do that. So, I think it's really important to give users a choice to know there's such application, and this thing is possible. I want to recreate the link. Is that ok? —Preceding unsigned comment added by Hossein.ir (talk • contribs) 14:18, 4 July 2006

Could you cite these FAQs please? Mis-read your comment. Thought you were saying the FAQs pointed to SP's recovery software. Never mind. AlistairMcMillan 16:49, 4 July 2006 (UTC)

Please read Wikipedia:External links. Particularly the section titled "What should not be linked". AlistairMcMillan 19:05, 4 July 2006 (UTC)

I looked at the stellarinfo site and found that the program only works with Windows! So, if you only have Linux, you'll have to install windows to recover you're data! -- Jdm64 18:56, 4 July 2006 (UTC)

I've read the Wikipedia guidlines, and I still can't see a problem in adding the link. Right now, it's the only way of recovering files on ext3. Do you think that this is promoting the product? I don't think so. Could I add the link? It may help some people in recovering their data. By the way, somebody considered me as a newbie, and filled my talk page with welcome message. I want to say that I'm not a newbie here, but most of my contributions goes into Persian Wikipedia, with the user name of حسین. —Preceding unsigned comment added by Hossein.ir (talk • contribs) 19:46, 7 July 2006

If you re-add the link it will likely be deleted, because of the policies linked above. You say it is the only way of recovering files on ext3, can you provided credible sources to back this up? And you say we should have the link because it "may help some people", but this is an encyclopedia not a HOWTO. AlistairMcMillan 13:14, 8 July 2006 (UTC)

Look at these links:

* Canberra Linux User Group Mailing List

* ext3 FAQ, Q:: How can I recover....

You're just linking to the policies page; Could you please discuss about it? I can't see a problem with adding that link. Thanks

Hossein.ir 15:10, 9 July 2006 (UTC)

Those links make it sound like any undeletion software is unlikely to work. Do you have any independent evidence that the product actually works? The company says its software does something that the ext3 developers say isn't possible. That's a bold claim that requires some support, I think.

At first glance it looks like the company wants users to pay $80-$200 for a Windows program in the desperate hope of recovering lost ext3 data. The website gives the impression that the program can recover any deleted information. That can't be true; what if the deleted inodes have been overwritten? Wmahan. 15:50, 9 July 2006 (UTC)

Perhaps Hossein you could take a minute to actually read the stellarinfo page. That would make a good start. Particularly this sentence: "Provides recovery of deleted file(s) for Ext2 File system only". Aside from this, our policy tends toward avoiding the addition of links that are intended solely to promote commercial products. That is the core of the problem. Also the fact that this product doesn't recover files on ext3 file systems... AlistairMcMillan 17:06, 9 July 2006 (UTC)

I haven't actually used the software, but it's available for evaluation download. I use Linux right now, and I can't experiment the application. And your question: I think all ther other undeletion programs have the same problem: overwriting data. And it seems that there's no application that can recover all deleted files, in all situations. Btw, look at this links. Does this seem fake to you? I can't say anything about those products. If someone has more information about file recovery, maybe we can add a section about recovering files. If you look at the article, the only tool that is provided for recovering files on ext3 filesystem is "grep"! Hossein.ir 17:11, 9 July 2006 (UTC)

That website is just a mirror of their advertising. It is not a review. And again, the people that created this product admit on their own website that it cannot recover files from ext3 volumes. Please stop going on about this. AlistairMcMillan 17:23, 9 July 2006 (UTC)

Plese Look at Wikipedia:Notability (software). With this guideline, I can not agree with you. This product has most of the criterias mentioned in the guideline.

Hossein.ir 14:22, 11 July 2006 (UTC)

OKAY, AGAIN, THIS SOFTWARE CANNOT RECOVER FILES FROM EXT3 VOLUMES. DELETED FILES ON EXT3 VOLUMES CANNOT BE RECOVERED WITH THIS SOFTWARE. EXT3 VOLUME FILES, WITH THIS SOFTWARE, CANNOT BE RECOVERED. THEY SAY THAT, QUITE CLEARLY, ON THEIR OWN WEBSITE. PLEASE READ THEIR WEBSITE: "Provides recovery of deleted file(s) for Ext2 File system only". AlistairMcMillan 16:39, 11 July 2006 (UTC)

Nope! You're absolutely wrong. Get the software, Install it, and tell me it doesn't work. I've used it and it works. What do you say then? And please don't lie ;-) Have you ever heard of google before? This is the first result of searching "stellar Linux":

Linux Data Recovery Software for Ext2, Ext3 Volume Recovery ...
Stellar Pheonix Linux - is fully automatic Linux data recovery software 
for the Ext2 and Ext3 File System volumes. 
Recovers data from Ext2 and Ext3 File ...

You're really careless. You don't even need to open the website. At least someone can use grep(read the faq). So don't tell "It's impossible", "It's impossible". And don't write it all caps. It's the way Internet trolls write. If you were in IRC, someone would probably ban you. So, take care. Please read the guideline I've mentioned before. Please read their website. Please try to do a little googling before speaking in this way. Please watch some famous Linux related website before claiming what works and what don't works. Hossein.ir 21:00, 12 July 2006 (UTC)

• Step 1: Open this page http://www.stellarinfo.com/linux-data-recovery.htm in a web browser.
• Step 2: Find the section header titled: "Key Features"
• Step 3: Find and read the third sentence that begins with the words "Provides recovery of deleted file(s)..." and ends with the words "...for Ext2 File system only."
• Step 4: ???
• Step 5: Profit!!!

AlistairMcMillan 21:38, 12 July 2006 (UTC)

I think the confusion stems from the fact that the link (www.stellarinfo.com/linux-data-recovery.htm) is unclear on exactly what can be recovered. I doubt that the ambiguity is entirely accidental; it probably benefits sales of the product, by increasing the number of users who hope to recover lost data.

The site says both of the following:

1. Stellar Phoenix Linux - is ... software for Ext2 and Ext3 File system volumes.
2. Provides recovery of deleted file(s) for Ext2 File system only.

Not having used the software, my impression is that it can recover blocks of data from Ext3 volumes, but not entire files. I don't think it is possible to recover files, given the FAQ that Hossein.ir linked above, which says that Ext3 "actually zeros out the block pointers in the inode" when deleting files.

My opinion is that the link should not be included until it is added by an editor who has used it and found it useful. After adding the link, Hossein.ir said "I haven't actually used the software". Wmahan. 22:14, 12 July 2006 (UTC)

Although you can read this in another part of the page:

I think they've talked of ext2, because they wanted to make a difference from their other tools that recover files from fat32/NTFS. But, I agree with Wmahan. AlistairMcMillan, are you proud enough to check this software, or you want to repeat "it's fake", "it's fake" all the time?

Hossein.ir 20:09, 13 July 2006 (UTC)

I have three words for you: no original research. AlistairMcMillan 21:13, 13 July 2006 (UTC)

As Wmahan mentioned, block pointers in the inodes are zeroed. But it does not actually zero the blocks, so it's possible to recover the files, at least in theory. And as I've read their website, they claim that their software has a "unique" scanning method, that finds deleted files, and recover them. As I remember, when the virus Chernobyl deleted many computers data by deleting partition tables, some guy had wrote a program that could recover the partition table, by scanning through the disk. You can find it here. Technically, I think this is feasible, because there's a chain of pointers, that you've lost one level of them. see inode for more information.

Hossein.ir 20:26, 13 July 2006 (UTC)

Btw, look at this open source software, Salvage. Please try it, and tell your opinion. It seems that including a link to this, is better from mentioning that propertiary product.

From its web page:

salvage first-block count-blocks

Hossein.ir

I'm sure I asked this before, but can you point to a single verifiable source on either of these programs? AlistairMcMillan 21:23, 13 July 2006 (UTC)

For sure Yes. The latter program is a patch to the latest version of e2fsprogs, by a usuall contributor to e2fsprogs, and I think it will be merged into upstream in next version(it's not now). Keld Simonsen, the author, is a usuall contributor to [e2fsprogs]. The program is a opensource software, that is available for downlaod, and testing; I've downloaded, compiled, and tested the program and it worked, but as as I know, for wikipedia this is not enough. But consider that Keld Simonsen, is a professional Linux programmer, and he has two RFCS. It seems that he's a notable person, and he deserves a page in wikipedia. I can not find a better way to make you understand This really works. If you were a expert in this area, we would have no problems. Do you have any experience in free software(ever heard of GNU??) and specially Linux?

For the previous program, there are a lot of well-known Linux-related website, that have listed this program.

• Softpedia
• Tucows
• IceWalkers

These were some famous websites from so many other sources. It's not original research.

Hossein.ir 22:16, 13 July 2006 (UTC)

Have you read our policies on sources? Those are software directories. Anyone can get software listed in a directory. We're looking for something like a magazine review... AlistairMcMillan 23:18, 13 July 2006 (UTC)

Ok, in this way, I can not even talk about the e2fsprogs itself. If I want to say that "there's a patch that does that thing" what should I do? Hey, we're talking about software. In the way that you think, even articles like GNOME and XFCE are invalid, because all the external links lead to their website, and their friends'. What about the Stellar Phoenix Linux? The sources seems to be enough.

BTW, when I explain the problem to you to show you it's feasible, and I give you links, you call it original research. Have you read the guideline yourself? It's mainly used for physics, and these kind of things. It seems that you've read it like the web pages of the Stellar Phoenix Linux(Just first few lines). Hey, I'm not researching. I'm giving you some links, to some well know websites, that introduce this software. If I find a magazine that contains an article, do you call it research....! I want to talk with another person about this issue. It seems that this is your POV.

Hossein.ir 00:25, 14 July 2006 (UTC)

I forgot to say, how can I find a source for a source or an external link for an external link? Have you thought of this? No external link has an extra external link to prove itself.

Hossein.ir 00:30, 14 July 2006 (UTC)

[edit] ext3 dirindex

What is ext3 dir-index about and shouldn't it be mentioned in the Ext3 article?--Hhielscher 20:11, 6 October 2006 (UTC)

dir_index is another name for H-Tree directory indexing. It's the name used by tune2fs, e.g. tune2fs -O dir_index /dev/hda1 . Yep I think it should be mentioned, I found the different names for the same thing confusing. Dj68 22:51, 16 November 2006 (UTC)

[edit] Defragmentation

There is a tool which can be used for ext3, "shake" :

• http://vleu.net/shake/
• http://forums.gentoo.org/viewtopic-t-463204-postdays-0-postorder-asc-start-0.html

Scullder 19:51, 8 December 2006 (UTC)

There is a tool from a Kerneldeveloper, just some script, he claims that it is "filesystem agnostic". For me it "just works (tm)" on ext3 while being mounted. It can be found here: http://ck.kolivas.org/apps/defrag/

"Is a lack of defrag tool a disadvantage? I mean: Does impact in a serious way the fragmentation on ext3? I don't think so, but..."

--Sebelk 01:50, 20 December 2006 (UTC)

In order to check the fragmentation on ext3 one can use the filefrag(1) utility. It is built above the FIBMAP and FIGETBSZ ioctls which which allow to enumerate the physical blocks used by a given file. As these ioctls can be implemented on various filesystems, filefrag(1) might work on something else than ext3 as well. Defragmenting is just copying the file data to a new file, checking whether it is less fragmented, and possibly replacing the original file.... Adam Mirowski (talk) 19:51, 13 April 2008 (UTC)

[edit] File timestamps resolution

Would it be relevant to mention that ext3 does not have sub-second timestamps resolution, in contrast with other file systems (NTFS?). I personally would have liked to find that information in Wikipedia, especially if it was comparative.

Rngadam 18:43, 15 January 2007 (UTC)

I think this is relevant. Please add...

[edit] Maximum File Size

Most people want to know the maximum file size of THEIR ext3 file system.

In order to do this, they must determine the block size of THEIR file system.

Currently, this entry provides a table that tells the max file size for various block sizes, but it offers little direction for people to determine THEIR ext3's maximum file size.

I wrote an article that tells people how to determine their block size in Linux: http://www.howtoadvice.com/Ext3Max/

Many people will find this link very helpful. Please consider it for linkage. —Preceding unsigned comment added by 209.0.204.89 (talk) 00:10, 3 October 2007 (UTC)

I've corrected the max file size for the 8KiB block size. The source document (http://www.charmed.com/txt/ext2.txt) says 2TiB and that is correct because it's limited by the i_blocks field which stores the size as 512byte blocks as a 32bit number. --Djm6809 (talk) 02:40, 20 November 2007 (UTC)

[edit] Maximum size stats

The table currently in the article contradicts [1] which says:

Ext3 can support files up to 1TB. With a 2.4 kernel the filesystem size is limited by the maximal block device size, which is 2TB. In 2.6 the maximum (32-bit CPU) limit is of block devices is 16TB, but ext3 supports only up to 4TB.

Can anyone determine what the truth is? The source which is referenced in the article now is much older than the FAQ, and is from the era before ext3 was available for Linux 2.4 kernels. -- Beland (talk) 18:18, 23 January 2008 (UTC)

Your article is outdated. Probably wikipedia should specify that 4K is by far the most common blocksize for PCs, and its limits are 16T for filesystems, and 2T for files. That's what most people care about, anyway.86.120.236.174 (talk) —Preceding comment was added at 06:20, 22 March 2008 (UTC)

Well, there are people who use 8k, 16k, and even larger filesystems. These only work on architectures such as the Itanium, Power PC, etc., where the page size is greater than 4k. (The block size must be less than or equal to the page size.) So this isn't common, but table in the filesystem does list the maximum filesystem size based on block size, which is a correct thing to do. I have received bug reports in e2fsprogs saying that things don't quite work right if you try to use a block size of 64k and you have an empty directory block, so there are people experimenting with block sizes up to 64k.Ted (talk) 15:30, 19 April 2008 (UTC)

[edit] Subdirectory limit

I've just been discussion ext3 limitations on IRC. Of particular interest is the following line in note #1:

The max number of subdirectories in one directory is fixed to 32000.

This is not entirely correct. The maximum number of hardlinks per inode is fixed at 32000. Every directory in the filesystem has one hardlink consumed for:

• the '.' special directory and
• the parent directory's link (if not /) and
• lost+found's '..' link (if the root of the mountable filesystem)

Thus, the maximum number of subdirectories you can have in any one directory varies between 31998 and 31997.

- Man in shack (talk) 13:32, 13 June 2008 (UTC)