Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

1
vote
2answers
145 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...
2
votes
1answer
1k views

How to add precedence to LALR parser like in YACC?

Please note, I am asking about writing LALR parser, not writing rules for LALR parser. What I need is... ...to mimic YACC precedence definitions. I don't know how it is implemented, and below I ...
0
votes
1answer
172 views

How lookaheads are propagated in “channel” method of building LALR parser?

The method is described in Dragon Book, however I read about it in ""Parsing Techniques" by D.Grune and C.J.H.Jacobs". I start from my understanding of building channels for NFA: channels are built ...
1
vote
1answer
267 views

How to read reduce/shift conflicts in LR(1) DFA?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when ...
3
votes
2answers
2k views

How to get lookahead symbol when constructing LR(1) NFA for parser?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial ...
3
votes
1answer
2k views

Extracting color profile information from JPEG files

I'm trying to look up info about reading JPEG's color profile info and to my surprise there's very little open specific how-to information on that regard, but rather lots of general explanation on ...
8
votes
4answers
281 views

How should I implement a command processing application?

I want to make a simple, proof-of-concept application (REPL) that takes a number and then processes commands on that number. Example: I start with 1. Then I write "add 2", it gives me 3. Then I ...
13
votes
5answers
454 views

How can I best manage making open source code releases from my company's confidential research code?

My company (let's call them Acme Technology) has a library of approximately one thousand source files that originally came from its Acme Labs research group, incubated in a development group for a ...
5
votes
1answer
584 views

Scripting custom drawing in Delphi application with IF/THEN/ELSE statements?

I'm building a Delphi application which displays a blueprint of a building, including doors, windows, wiring, lighting, outlets, switches, etc. I have implemented a very lightweight script of my own ...
3
votes
1answer
205 views

What does it mean that StAX parsers aren't complete or correct?

I've been browsing the net and found theses two slides on site of XOM (XML object model): slide1 slide2 Where they refer to SAX(push) and StAX(pull) parsers. So I'm wondering, what does it mean ...
17
votes
4answers
1k views

Generic rule parser for RPG board game rules - how to do it?

I want to build a generic rule parser for pen and paper style RPG systems. A rule can involve usually 1 to N entities 1 to N roles of a dice and calculating values based on multiple attributes of an ...
8
votes
2answers
1k views

Persisting natural language processing parsed data

I've recently started experimenting with natural language processing (NLP) using Stanford's CoreNLP, and I'm wondering what are some of the standard ways to store NLP parsed data for something like a ...
3
votes
3answers
543 views

design for a parser to handle very large files

I have written a program which records protocol messages between an application and a hardware device which matches each application request with each hardware response. This is so that I can later ...
1
vote
1answer
304 views

Does JAXP natively parse HTML?

So, I whip up a quick test case in Java 7 to grab a couple of elements from random URIs, and see if the built-in parsing stuff will do what I need. Here's the basic setup (with exception handling etc ...
10
votes
5answers
8k views

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, ...
3
votes
3answers
705 views

How do I translate user input into a fictitious language?

For experimental reasons, I am trying to convert user input into a fictitious language. All of the translation can be 1:1. I would prefer if I could accomplish this with PHP. Should I use gettext ...
1
vote
1answer
146 views

How do I parse a header with two different version [ID3] avoiding code duplication?

I really hope you can give me some interesting viewpoints for my situation, because I am not satisfied with my current approach. I am writing an MP3 parser, starting with an ID3v2 parser. Right now ...
2
votes
2answers
154 views

parsing terminology: comments+whitespaces vs actual code

In languages like c/c++ spacing and comments are ignored and only actual code gets into compiler. I'm interested if there is accepted way of naming these two things? comments & spacing ...
5
votes
4answers
1k views

Programming Language Parser (in Java) - What would be a better design alternative for a special case?

Background I'm currently designing my own programming language as a research project. I have most of the grammar done and written down as context-free grammar, and it should be working as is. - Now ...
3
votes
1answer
215 views

How to support mixed grammars?

I am currently writing a grammar for a programming language (PowerBuilder) and I am also collecting documentation about parsing and compilers creation. Now I would like to add in my grammar (that ...
7
votes
2answers
314 views

Parsing multiple file formats/protocols

We are starting a project where we will need to write parsers for a bunch of binary file formats, each of them representing very similar data (time-value series from different measurement devices). ...
1
vote
2answers
1k views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
7
votes
4answers
619 views

How are comments expressed in programming language grammars?

I'm learning how to build parsers using grammars, but I got stuck trying to express comments, because they can appear almost anywhere. This indicates that comments can be stripped from the token ...
0
votes
2answers
213 views

Using a parser to locate faulty code

Lately I've been working a lot in PHP and have run into an abnormally large number of parsing errors. I realize these are my own fault and a result of sloppy initial coding on my part, but it's ...
9
votes
5answers
22k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
3
votes
2answers
2k views

Is there a more modern program than lex or yacc, which does not require JVM?

Is there a "modern" bison and flex equivalent which does not require the JVM or .NET or similar "heavy" runtimes? Thanks for comments, edit: By modern, I mean for example Antlr and another one in ...
8
votes
6answers
3k views

Best way to parse a file

I'm trying to find a better solution for making a parser to some of the famous file formats out there such as: EDIFACT and TRADACOMS. If you aren't familiar with these standards then check out this ...
5
votes
2answers
1k views

What is this algorithm for converting strings into numbers called?

I've been doing some work in Parsec recently, and for my toy language I wanted multi-based fractional numbers to be expressible. After digging around in Parsec's source a bit, I found their ...
0
votes
1answer
173 views

What are known approaches to graphing algebraic expressions?

I am planning to build an expression parser that will be used to graph algebraic functions ( think TI-83 ) with JavaScript. Functions will take the form of f(x)= Aside from typical operators such ...
8
votes
6answers
4k views

What is the simplest human readable configuration file format? [closed]

Current configuration file is as follows: mainwindow.title = 'test' mainwindow.position.x = 100 mainwindow.position.y = 200 mainwindow.button.label = 'apply' mainwindow.button.size.x = 100 ...
-1
votes
2answers
466 views

How do I capture information from a website that doesn't provide an API?

Do you know any good tutorials, frameworks, anything that can help me to write code that captures information from a website that don't have a public API, or hasn't been written in a RESTful way? ...
1
vote
1answer
746 views

Browser parsers vs Stand-alone parsers

Firefox/Chromium have well implemented(?) parsers. Then why are parsers like lxml and other such parsers being developed with so much effort put into it? Aren't they re-inventing the wheel? Or are ...
3
votes
1answer
715 views

How can I test a parser for a bespoke XML schema?

I'm parsing a bespoke XML format into an object graph using .NET 4.0. My parser is using the System.XML namespace internally, I'm then interrogating the relevant properties of XmlNodes to create my ...
10
votes
5answers
2k views

Coming up with tokens for a lexer

I'm writing a parser for a markup language that I have created (writing in python, but that's not really relevant to this question -- in fact if this seems like a bad idea, I'd love a suggestion for a ...
6
votes
2answers
591 views

Parser combinator that looks like BNF

Is it possible to construct a parser combinator library that reads like a BNF grammar? I don't know of any, so I started wondering if there are reasons it's impossible or undesirable to do so. It ...
13
votes
2answers
2k views

What's the simplest example out there to explain the difference between Parse Trees and Abstract Syntax Trees?

To my understanding, a parser creates a parse tree, and then discards it thereafter. However, it can also pop out an abstract syntax tree, which the compiler supposedly makes use of. I'm under the ...
12
votes
5answers
3k views

Are separate parsing and lexing passes good practice with parser combinators?

When I began to use parser combinators my first reaction was a sense of liberation from what felt like an artificial distinction between parsing and lexing. All of a sudden everything was just ...
23
votes
6answers
1k views

What are the arguments against parsing the Cthulhu way?

I have been assigned the task of implementing a Domain Specific Language for a tool that may become quite important for the company. The language is simple but not trivial, it already allows nested ...
17
votes
12answers
6k views

How to write a command interpreter/parser?

Problem: Run commands in the form of a string. command example: /user/files/ list all; equivalent to: /user/files/ ls -la; another one: post tw fb "HOW DO YOU STOP THE TICKLE MONSTER?;" ...
2
votes
1answer
457 views

Idea of an algorithm to detect a website's navigation structure?

Currently I am in the process of developing an importer of any existing, arbitrary (static) HTML website into the upcoming release of our CMS. While the downloading the files is solved successfully, ...
4
votes
3answers
675 views

Any good reason to open files in text mode?

(Almost-)POSIX-compliant operating systems and Windows are known to distinguish between 'binary mode' and 'text mode' file I/O. While the former mode doesn't transform any data between the actual file ...
5
votes
1answer
243 views

Is there such thing as an example driven parser generator or ad-hoc DSL development?

I'm intrested to know if there exists a tool that lets you input examples of valid documents and lets you then generalize from that to a reusable parser. I can imagine this, but everytime i start ...
6
votes
3answers
5k views

What programming language is most suitable for handling unstructured data?

I'm trying to automate the application of metadata to huge amount of text, but I'm not sure what language would make this task easier (if there is one). What programming language is most suitable ...
2
votes
1answer
270 views

Best way to parse optional grammar rules?

I started to write a hand written parser just for fun. For simple rules it is working, but I have problem with the optional grammar rules. I marked the optional rules with a question mark. Here is a ...
2
votes
2answers
4k views

Fastest C++ XML parsing library

I have thousands of .xml files from size 1MB-45MB (no DTDs). I need to parse and further manipulate these XML files before generating separate .xml files with the results of my regex. What the ...
6
votes
3answers
820 views

C++ XML Parsing: Suggestions on Approach for Parsing and Storing data

I am looking into developing a C++ application to parse xml (using the rapidxml framework), and I would like some advice on how to approach this. The file I want to parse is a XML game file that ...
7
votes
2answers
3k views

Algorithm for formating SQL code

I need a tool (for in house usage) that will format SQL code (SQL Server/MySQL). There are various 3rd party tools and online web sites that do it but no exactly how I need it. So I want to write my ...
10
votes
3answers
2k views

How should I specify a grammar for a parser?

I have been programming for many years, but one task that still takes me inordinately long is to specify a grammar for a parser, and even after this excessive effort, I'm never sure that the grammar ...
6
votes
2answers
410 views

Idea for a domain specific language or DLR port?

I have my undergraduate final year project coming up and am very interested in lexers, parsers, compilers and so on. I would like to use the DLR (.NET 4.0 dynamic language runtime) for my ...
5
votes
3answers
2k views

Parsing scripts that use curly braces

To get an idea of what I'm doing, I am writing a python parser that will parse directx .x text files. The problem I have deals with how the files are formatted. Although I'm writing it in python, I'm ...