Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

3
votes
3answers
585 views

How do I translate user input into a fictitious language?

For experimental reasons, I am trying to convert user input into a fictitious language. All of the translation can be 1:1. I would prefer if I could accomplish this with PHP. Should I use gettext ...
1
vote
1answer
138 views

How do I parse a header with two different version [ID3] avoiding code duplication?

I really hope you can give me some interesting viewpoints for my situation, because I am not satisfied with my current approach. I am writing an MP3 parser, starting with an ID3v2 parser. Right now ...
2
votes
2answers
152 views

parsing terminology: comments+whitespaces vs actual code

In languages like c/c++ spacing and comments are ignored and only actual code gets into compiler. I'm interested if there is accepted way of naming these two things? comments & spacing ...
5
votes
4answers
967 views

Programming Language Parser (in Java) - What would be a better design alternative for a special case?

Background I'm currently designing my own programming language as a research project. I have most of the grammar done and written down as context-free grammar, and it should be working as is. - Now ...
3
votes
1answer
193 views

How to support mixed grammars?

I am currently writing a grammar for a programming language (PowerBuilder) and I am also collecting documentation about parsing and compilers creation. Now I would like to add in my grammar (that ...
7
votes
2answers
288 views

Parsing multiple file formats/protocols

We are starting a project where we will need to write parsers for a bunch of binary file formats, each of them representing very similar data (time-value series from different measurement devices). ...
1
vote
2answers
995 views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
6
votes
4answers
475 views

How are comments expressed in programming language grammars?

I'm learning how to build parsers using grammars, but I got stuck trying to express comments, because they can appear almost anywhere. This indicates that comments can be stripped from the token ...
0
votes
2answers
206 views

Using a parser to locate faulty code

Lately I've been working a lot in PHP and have run into an abnormally large number of parsing errors. I realize these are my own fault and a result of sloppy initial coding on my part, but it's ...
9
votes
5answers
14k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
1
vote
2answers
2k views

Is there a more modern program than lex or yacc, which does not require JVM?

Is there a "modern" bison and flex equivalent which does not require the JVM or .NET or similar "heavy" runtimes? Thanks for comments, edit: By modern, I mean for example Antlr and another one in ...
6
votes
6answers
2k views

Best way to parse a file

I'm trying to find a better solution for making a parser to some of the famous file formats out there such as: EDIFACT and TRADACOMS. If you aren't familiar with these standards then check out this ...
5
votes
2answers
1k views

What is this algorithm for converting strings into numbers called?

I've been doing some work in Parsec recently, and for my toy language I wanted multi-based fractional numbers to be expressible. After digging around in Parsec's source a bit, I found their ...
0
votes
1answer
141 views

What are known approaches to graphing algebraic expressions?

I am planning to build an expression parser that will be used to graph algebraic functions ( think TI-83 ) with JavaScript. Functions will take the form of f(x)= Aside from typical operators such ...
6
votes
6answers
3k views

What is the simplest human readable configuration file format? [closed]

Current configuration file is as follows: mainwindow.title = 'test' mainwindow.position.x = 100 mainwindow.position.y = 200 mainwindow.button.label = 'apply' mainwindow.button.size.x = 100 ...
-1
votes
2answers
351 views

How do I capture information from a website that doesn't provide an API?

Do you know any good tutorials, frameworks, anything that can help me to write code that captures information from a website that don't have a public API, or hasn't been written in a RESTful way? ...
-1
votes
1answer
620 views

Browser parsers vs Stand-alone parsers

Firefox/Chromium have well implemented(?) parsers. Then why are parsers like lxml and other such parsers being developed with so much effort put into it? Aren't they re-inventing the wheel? Or are ...
3
votes
1answer
544 views

How can I test a parser for a bespoke XML schema?

I'm parsing a bespoke XML format into an object graph using .NET 4.0. My parser is using the System.XML namespace internally, I'm then interrogating the relevant properties of XmlNodes to create my ...
9
votes
5answers
2k views

Coming up with tokens for a lexer

I'm writing a parser for a markup language that I have created (writing in python, but that's not really relevant to this question -- in fact if this seems like a bad idea, I'd love a suggestion for a ...
6
votes
2answers
506 views

Parser combinator that looks like BNF

Is it possible to construct a parser combinator library that reads like a BNF grammar? I don't know of any, so I started wondering if there are reasons it's impossible or undesirable to do so. It ...
10
votes
2answers
2k views

What's the simplest example out there to explain the difference between Parse Trees and Abstract Syntax Trees?

To my understanding, a parser creates a parse tree, and then discards it thereafter. However, it can also pop out an abstract syntax tree, which the compiler supposedly makes use of. I'm under the ...
11
votes
5answers
2k views

Are separate parsing and lexing passes good practice with parser combinators?

When I began to use parser combinators my first reaction was a sense of liberation from what felt like an artificial distinction between parsing and lexing. All of a sudden everything was just ...
23
votes
6answers
1k views

What are the arguments against parsing the Cthulhu way?

I have been assigned the task of implementing a Domain Specific Language for a tool that may become quite important for the company. The language is simple but not trivial, it already allows nested ...
16
votes
12answers
5k views

How to write a command interpreter/parser?

Problem: Run commands in the form of a string. command example: /user/files/ list all; equivalent to: /user/files/ ls -la; another one: post tw fb "HOW DO YOU STOP THE TICKLE MONSTER?;" ...
2
votes
1answer
378 views

Idea of an algorithm to detect a website's navigation structure?

Currently I am in the process of developing an importer of any existing, arbitrary (static) HTML website into the upcoming release of our CMS. While the downloading the files is solved successfully, ...
4
votes
3answers
565 views

Any good reason to open files in text mode?

(Almost-)POSIX-compliant operating systems and Windows are known to distinguish between 'binary mode' and 'text mode' file I/O. While the former mode doesn't transform any data between the actual file ...
4
votes
1answer
207 views

Is there such thing as an example driven parser generator or ad-hoc DSL development?

I'm intrested to know if there exists a tool that lets you input examples of valid documents and lets you then generalize from that to a reusable parser. I can imagine this, but everytime i start ...
4
votes
3answers
2k views

What programming language is most suitable for handling unstructured data?

I'm trying to automate the application of metadata to huge amount of text, but I'm not sure what language would make this task easier (if there is one). What programming language is most suitable ...
2
votes
1answer
237 views

Best way to parse optional grammar rules?

I started to write a hand written parser just for fun. For simple rules it is working, but I have problem with the optional grammar rules. I marked the optional rules with a question mark. Here is a ...
2
votes
2answers
3k views

Fastest C++ XML parsing library

I have thousands of .xml files from size 1MB-45MB (no DTDs). I need to parse and further manipulate these XML files before generating separate .xml files with the results of my regex. What the ...
6
votes
3answers
659 views

C++ XML Parsing: Suggestions on Approach for Parsing and Storing data

I am looking into developing a C++ application to parse xml (using the rapidxml framework), and I would like some advice on how to approach this. The file I want to parse is a XML game file that ...
7
votes
2answers
2k views

Algorithm for formating SQL code

I need a tool (for in house usage) that will format SQL code (SQL Server/MySQL). There are various 3rd party tools and online web sites that do it but no exactly how I need it. So I want to write my ...
10
votes
3answers
1k views

How should I specify a grammar for a parser?

I have been programming for many years, but one task that still takes me inordinately long is to specify a grammar for a parser, and even after this excessive effort, I'm never sure that the grammar ...
6
votes
2answers
388 views

Idea for a domain specific language or DLR port?

I have my undergraduate final year project coming up and am very interested in lexers, parsers, compilers and so on. I would like to use the DLR (.NET 4.0 dynamic language runtime) for my ...
4
votes
3answers
2k views

Parsing scripts that use curly braces

To get an idea of what I'm doing, I am writing a python parser that will parse directx .x text files. The problem I have deals with how the files are formatted. Although I'm writing it in python, I'm ...
2
votes
3answers
2k views

Writing a CSS parser in C#. What do you think is the best strategy?

I'm in the middle of writing a CSS parser in C#. I'm well under way, but I also have those times where I wonder if I'm taking the best approach. The things I've considered are: Feed the CSS grammar ...
1
vote
5answers
841 views

I need to develop a parser. Can I use Lex and Yacc for the purpose?

I need to extract very particular data from log files(of different types and formats). Since I am a recent college passout; my mind ran to using Lex and Yacc for the purpose. Now I have the following ...
7
votes
1answer
242 views

Should I let my users write BnfExpressions to extend my grammar?

Preface I'm designing a templating language (please skip the don't/why?? speech). One of the major goals of this language is to be extensible. There are 2 main elements in my language. "Tags" and ...
3
votes
1answer
154 views

Performance issues when parsing code [closed]

I was wondering about performance issues when parsing a source file that is being edited by the user (for example, when you need to give a syntax highlight). I think that the simplest approach is to ...
8
votes
6answers
3k views

Techniques for parsing XML

I've always found XML somewhat cumbersome to process. I'm not talking about implementing an XML parser: I'm talking about using an existing stream-based parser, like a SAX parser, which processes the ...
9
votes
5answers
2k views

unit tests for a csv parser

What tests should I use to unit test a csv parser? I have a simple csv parser in C#, and I want to be sure I have good unit test coverage of all the common (and uncommon) edge cases. What tests ...
4
votes
5answers
818 views

What would happen if you defined your system's CSV delimiter as being a quotation mark?

Title says it all. If the CSV's system delimiter was " (as opposed to a comma or pipe or other common alternatives), how would anything deal with it? The crux of the matter is of course that by ...
2
votes
3answers
2k views

c++ write own xml parser vs using tinyxml

I am currently in a task to generate an XML file for an srt text file containing timestamps and corresponding text. To generate an exe file which accepts file name input and outputs the relevant XML ...
7
votes
3answers
3k views

What issues tend to arise when working with HL7 messages?

I'm testing a product for health care businesses, and we're working with HL7 messages. I saw people groaning on another question about the issues with HL7 but not mentioning specifics. Can someone ...
40
votes
12answers
9k views

Should I use a parser generator or should I roll my own custom lexer and parser code?

What specific advantages and disadvantages of each way to working on a programming language grammar? Why/When should I roll my own? Why/When should I use a generator?