The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
4answers
680 views

Is it safe to convert Windows file paths to Unix file paths with a simple replace?

So for example say I had it so that all of my files will be transferred from a windows machine to a unix machine as such: C:\test\myFile.txt to {somewhere}/test/myFile.txt (drive letter is irrelevant ...
3
votes
1answer
140 views

How to store the file names, start offset and length while avoiding the issue of self imposed limits (lookup table) or having to scan the entire file?

I am attempting to learn more about C and it's descendants(C++ mainly). I have decided that I would like to create a "file system" of sorts. Not a particularly advanced one mind you but something to ...
1
vote
6answers
499 views

Why can we not insert into files without the additional writes? (I neither mean append, nor over-write)

This occurs as a programming language independent problem to me. I have a file with the content aaabddd When I want to insert C behind b then my code needs to rewrite ddd to get aaabCddd Why ...
1
vote
2answers
243 views

Why must directories be empty before being deleted?

As far as I know, deleting a non empty directory could work the same way as deleting an empty directory: by removing the pointer to the directory's metadata there would be no pointers to the items it ...
5
votes
1answer
185 views

Why is Java 7's new file API usage still so scarce? [closed]

I don't even know whether this is the correct StackExchange site to ask such a question but here it is... I principally lurk on StackOverflow; and I am frankly flabbergasted by the number of ...
1
vote
0answers
129 views

What database structure is suitable for tracking File audits?

I need to track permissions and access requests to a file server in a database. I'm given the full path of the folder and am considering parsing the path (splitting on the "/" character) and creating ...
2
votes
2answers
271 views

Why is it not standard to implement abstraction layers for the file system?

I have been taught to access databases through abstraction layers. I was wondering why it is not also standard practice to access the file system through an abstraction layer? It seems to me unit ...
1
vote
2answers
3k views

Is it wise to store a big lump of json on a database row

I have this project which stores product details from amazon into the database. Just to give you an idea on how big it is: [{"title":"Genetic Engineering (Opposing ...
7
votes
4answers
483 views

What exactly does it mean that storing “large blobs in the database reduces performance”?

To someone who knows database internals this may be an easy question, but can someone explain in a clear way why storing large blobs (say 400 MB movies) in the database is supposed to decrease ...
5
votes
3answers
1k views

How many threads should access the file system at the same time?

We have a module in an application which stores data in multiple files and multilevel directories and access them from multiple threads (both reads and writes). The directory structure is based on a ...
1
vote
2answers
885 views

Finding duplicate files? [duplicate]

I am going to be developing a program that detects duplicate files and I was wondering what the best/fastest method would be to do this? I am more interested in what the best hash algorithm would be ...
1
vote
0answers
239 views

Watching file changes/additions/removal, but with an eye on partial transfer

I would like to monitor the filesystem in python, so that my application gets warned of the new file addition, file removal, or file change. Once the file is detected, the application starts ...
3
votes
2answers
528 views

Is it a bad idea to sync file system with remote server using HTTP?

I have started a project which will duplicate Dropbox or Google Drive behavior but using Amazon S3 az a backend. Idea is very simple, a Node.js server that watchs a directory for file changes and PUT ...
8
votes
2answers
29k views

How to detect the encoding of a file?

On my filesystem (Windows 7) I have some text files (These are SQL script files, if that matters). When opened with Notepad++, in the "Encoding" menu some of them are reported to have an encoding of ...
1
vote
5answers
423 views

Is it plausible to use a filesystem-based configuration format rather than an INI file?

After having had endless issues with INI-style configuration files and parsing them correctly (let along getting binary data right), a crazy idea crossed my mind recently: What if an INI file like # ...
2
votes
4answers
746 views

Best practice to sync long paths to filesystems where path length is limited?

I wrote a sync tool to synchronize folders/files from Alfresco to Windows. PROBLEM: In Alfresco, /files/can/have/very/very/very/long/filepaths/like/this.txt The Windows API prevents me from creating ...
4
votes
3answers
2k views

Lowercase in Linux file names

As I find that UpperCase are really readable for first letter word separation in long complex names, I tend to give some of my Linux files names with some UpperCase. Mostly executables, some ...
0
votes
3answers
365 views

Which language is more suitable heavy file tasks?

I need to write a script (based on basic functions) to process /image/audio/video files. The process is mainly filesystem tasks and converts. The database of files has been stored by mysql. The script ...
3
votes
3answers
211 views

Storing lots of large strings with frequent “appends” and few reads

In my current project, I need to store a very long ASCII string to each instance of a given object. This string will receive an 2 appends per minute and will not be retrieved so frequently. The ...
1
vote
1answer
561 views

NoSQL as file meta database

I am trying to implement a virtual file system structure in front of an object storage (Openstack). For availability reasons we initially chose Cassandra, however while designing file system data ...
2
votes
1answer
592 views

How to organize my site's file system properly?

Doing some reading on Stack Overflow, I've found a lot of information suggesting that proper organization of a file system is crucial to a well-written web app. One of the key pieces of evidence is ...
3
votes
1answer
2k views

File system implementation in MongoDB with GridFS

I am working on two projects that will both implement a Webdav server backed by a MongoDB GridFS. In each case, there is the potential for the system to store tens of millions of files spread across ...
2
votes
3answers
212 views

Is there an established convention for separating Windows file names in a string?

I have a function which needs to output a string containing a list of file paths. I can choose the separation character but I cannot change the data type (e.g. I cannot return a List<string> or ...
2
votes
1answer
589 views

Directory structure (file system design)

I was looking at how file systems are designed and noticed that most places say that the directory hierarchy can be implemented using a hash table. Could someone please explain me how using a hash ...
2
votes
1answer
90 views

concurrency semantics reading a file being written (ext)

In Ext 3/4 filesystem, if I have a log file open and being appended to, if another process tries to read it (e.g. with cat), what happens? Can the reading process read part of / an incomplete write? ...
1
vote
1answer
325 views

HTML5 file API or Java bridge to acces to local Files?

I've to access to files on the software user and I don't know if it's better to use a full JS app with HTML5 File Api rules or use Java and communicate with it ?
2
votes
7answers
2k views

Automatically delete files after they expire

I've got this idea for some time and I was wondering if anyone has seen such a feature/app in any operating system and if you haven't, what do you think about it. Where do you think I should begin? ...
2
votes
4answers
253 views

Where can I learn about how programs handle file systems like NTFS?

I'm writing a program to handle the xbox 360's STFS files. I'm encountering all sorts of problems about how to keep track of used vs unused blocks, reading non-sequential files, etc. These aren't so ...
4
votes
5answers
4k views

How to Model a simple file-system by UML class diagram

I want to model a file system which contains both files and directories, and directories can contain either files or other directories. This is what I have reached so far: In OOSE book, however, a ...
1
vote
6answers
187 views

Ways of marking changes being made to a file

I am looking for a lightweight way to notate changes made to a file. The obvious answer to this is a version control system. But this is more robust than necessary for what I would like to do. All I ...
9
votes
5answers
287 views

Examples of limitations in IT due to different bit length by design

I am teaching the course "Introduction in Programming" for the first-year students and would like to find interesting examples where the datatype size in bits, chosen by design, led to certain known ...
3
votes
3answers
179 views

Filesystems that use logging: If you're writing the data in the log (on disk), and in the actual locations themselves (also on disk) then…?

Aren't you essentially writing the same data twice, to the disk? Doesn't this cause a slowdown of a factor of ~2x? What optimizations can be made to minimize the cost of having to write things twice? ...
2
votes
3answers
641 views

Scalable distributed file system for blobs like images and other documents [closed]

Cassandra & HBase both do not efficiently support storage of blobs like images. Storing directly on HDFS stresses the Namenode because of huge number of files. Facebook uses Haystack for images ...
26
votes
11answers
2k views

Why are there so many competing filesystem designs? [closed]

Just a quick question, but why are there so many file systems still competing and in use today? (ntfs, fat32, ext3(ffs), etc) It seems that file system designers could agree upon the best aspects of ...
4
votes
2answers
224 views

Is there a fundamental reason for keeping file systems and versioning systems separate?

I've done some work looking at Next3 and ext3cow, and a few other file systems that provide snapshots and/or file versioning. It seems a bit strange to me that file systems only go part-way with ...
8
votes
4answers
452 views

File shredder algorithm

As a part of learning system programming, I am looking to implement a file shredder. The simplest way (and probably seen as naive) would be to replace the data bytes with zeroes (I know OS splits the ...
13
votes
5answers
1k views

Why is filesystem preferred for logs instead of RDBMS?

Question should be clear from its title. For example Apache saves its access and error logs in files instead of RDBMS no matter on how large or small scale it is being utilized. For RDMS we just have ...
5
votes
4answers
1k views

Why is Tortoise SVN case sensitive?

I've recently encountered this using TortoiseSVN, but I assume it will be the same for CVS based programs (correct me?). Out of pure curiosity, is there any reason why the CVS filesystem is ...
4
votes
7answers
736 views

How do i speedily traverse a file system while extracting/extrapolating various data and provide user feedback?

I'm working on a system file scanner that reveals info about various files (e.g. size, last used, duplicates, etc). Currently I'm traversing the file system once just to get a good measure of the ...
2
votes
4answers
512 views

Can memristors change programming paradigms? In what way?

Think about persistent storage that can preserve pointers: among other things there is no need to serialize/deserialize data. For example, you typically build a tree structure and then serialize it ...