Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

9
votes
4answers
740 views

Using a “dead man's switch” to manage time-sensitive code

In our software environment, we often run a/b tests, as is probably good practice. However, our environment is set up such that, in very short order, the code starts to become very crufty with dead ...
3
votes
2answers
6k views

First and Follow Sets for a Grammar [closed]

I'm studying for a Compiler Construction module I'm doing and I have a sample question as follows: Calculate the FIRST and FOLLOW sets for the following grammar.. S -> uBDz B -> Bv B -> w D ...
3
votes
1answer
243 views

Showing a grammar is ambiguous

I have the following question taken from a compilers course exam: Show that the following grammar is ambiguous. S = XcY X = a Y = b | Z Z = bW W = d | ϵ I drew the following tree: Am I correct ...
1
vote
2answers
206 views

How to extract operators from the grammar productions for conflict resolution in LALR parser?

Is there some standardized or widely accepted algorithm for picking up operators in shift/reduce conflicts in LALR parser? The question is naive, my problem is not with implementing my solution, but ...
2
votes
1answer
197 views

REST efficiency on count and sum methods

For example, if I have a Customer Transactions Table and I create a REST for it. The fields are: date, description, and total_amount. I am trying to figure out which one is much more efficient when ...
7
votes
2answers
480 views

Language parsing to find important words

I'm looking for some input and theory on how to approach a lexical topic. Let's say I have a collection of strings, which may just be one sentence or potentially multiple sentences. I'd like to ...
6
votes
2answers
508 views

What is a real-world use case of using a Chomsky Type-I (context-sensitive) grammar

I have been having some fun lately exploring the development of language parsers in the context of how they fit into the Chomsky Hierarchy. What is a good real-world (ie not theoretical) example of a ...
1
vote
2answers
122 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...
2
votes
1answer
501 views

How to add precedence to LALR parser like in YACC?

Please note, I am asking about writing LALR parser, not writing rules for LALR parser. What I need is... ...to mimic YACC precedence definitions. I don't know how it is implemented, and below I ...
0
votes
1answer
128 views

How lookaheads are propagated in “channel” method of building LALR parser?

The method is described in Dragon Book, however I read about it in ""Parsing Techniques" by D.Grune and C.J.H.Jacobs". I start from my understanding of building channels for NFA: channels are built ...
1
vote
1answer
164 views

How to read reduce/shift conflicts in LR(1) DFA?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when ...
3
votes
2answers
956 views

How to get lookahead symbol when constructing LR(1) NFA for parser?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial ...
2
votes
1answer
268 views

LL(∞) and left-recursion [closed]

I want to understand the relation between LL/LR grammars and the left-recursion problem (for any question I know parcially the answer, but I ask them as I don't know nothing, because I am a little ...
3
votes
1answer
691 views

Extracting color profile information from JPEG files

I'm trying to look up info about reading JPEG's color profile info and to my surprise there's very little open specific how-to information on that regard, but rather lots of general explanation on ...
8
votes
4answers
257 views

How should I implement a command processing application?

I want to make a simple, proof-of-concept application (REPL) that takes a number and then processes commands on that number. Example: I start with 1. Then I write "add 2", it gives me 3. Then I ...
13
votes
5answers
396 views

How can I best manage making open source code releases from my company's confidential research code?

My company (let's call them Acme Technology) has a library of approximately one thousand source files that originally came from its Acme Labs research group, incubated in a development group for a ...
5
votes
1answer
351 views

Scripting custom drawing in Delphi application with IF/THEN/ELSE statements?

I'm building a Delphi application which displays a blueprint of a building, including doors, windows, wiring, lighting, outlets, switches, etc. I have implemented a very lightweight script of my own ...
3
votes
1answer
187 views

What does it mean that StAX parsers aren't complete or correct?

I've been browsing the net and found theses two slides on site of XOM (XML object model): slide1 slide2 Where they refer to SAX(push) and StAX(pull) parsers. So I'm wondering, what does it mean ...
17
votes
4answers
933 views

Generic rule parser for RPG board game rules - how to do it?

I want to build a generic rule parser for pen and paper style RPG systems. A rule can involve usually 1 to N entities 1 to N roles of a dice and calculating values based on multiple attributes of an ...
8
votes
2answers
494 views

Persisting natural language processing parsed data

I've recently started experimenting with natural language processing (NLP) using Stanford's CoreNLP, and I'm wondering what are some of the standard ways to store NLP parsed data for something like a ...
3
votes
3answers
347 views

design for a parser to handle very large files

I have written a program which records protocol messages between an application and a hardware device which matches each application request with each hardware response. This is so that I can later ...
1
vote
1answer
206 views

Does JAXP natively parse HTML?

So, I whip up a quick test case in Java 7 to grab a couple of elements from random URIs, and see if the built-in parsing stuff will do what I need. Here's the basic setup (with exception handling etc ...
10
votes
5answers
3k views

Can the csv format be defined by a regex?

A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, ...
3
votes
3answers
445 views

How do I translate user input into a fictitious language?

For experimental reasons, I am trying to convert user input into a fictitious language. All of the translation can be 1:1. I would prefer if I could accomplish this with PHP. Should I use gettext ...
0
votes
1answer
123 views

How do I parse a header with two different version [ID3] avoiding code duplication?

I really hope you can give me some interesting viewpoints for my situation, because I am not satisfied with my current approach. I am writing an MP3 parser, starting with an ID3v2 parser. Right now ...
2
votes
2answers
150 views

parsing terminology: comments+whitespaces vs actual code

In languages like c/c++ spacing and comments are ignored and only actual code gets into compiler. I'm interested if there is accepted way of naming these two things? comments & spacing ...
5
votes
4answers
795 views

Programming Language Parser (in Java) - What would be a better design alternative for a special case?

Background I'm currently designing my own programming language as a research project. I have most of the grammar done and written down as context-free grammar, and it should be working as is. - Now ...
3
votes
1answer
157 views

How to support mixed grammars?

I am currently writing a grammar for a programming language (PowerBuilder) and I am also collecting documentation about parsing and compilers creation. Now I would like to add in my grammar (that ...
1
vote
1answer
431 views

Picture Parsing [closed]

If I open a picture file, lets say with an PNG extension, I will see bunch of code. Now let say I want to get some information from the picture mechanically. So the question here is what is the first ...
7
votes
2answers
261 views

Parsing multiple file formats/protocols

We are starting a project where we will need to write parsers for a bunch of binary file formats, each of them representing very similar data (time-value series from different measurement devices). ...
1
vote
2answers
791 views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
6
votes
4answers
371 views

How are comments expressed in programming language grammars?

I'm learning how to build parsers using grammars, but I got stuck trying to express comments, because they can appear almost anywhere. This indicates that comments can be stripped from the token ...
0
votes
2answers
201 views

Using a parser to locate faulty code

Lately I've been working a lot in PHP and have run into an abnormally large number of parsing errors. I realize these are my own fault and a result of sloppy initial coding on my part, but it's ...
7
votes
5answers
8k views

Getting data from a webpage in a stable and efficient way

Recently I've learned that using a regex to parse the HTML of a website to get the data you need isn't the best course of action. So my question is simple: What then, is the best / most efficient and ...
0
votes
2answers
1k views

Is there a more modern program than lex or yacc, which does not require JVM?

Is there a "modern" bison and flex equivalent which does not require the JVM or .NET or similar "heavy" runtimes? Thanks for comments, edit: By modern, I mean for example Antlr and another one in ...
6
votes
6answers
1k views

Best way to parse a file

I'm trying to find a better solution for making a parser to some of the famous file formats out there such as: EDIFACT and TRADACOMS. If you aren't familiar with these standards then check out this ...
5
votes
2answers
775 views

What is this algorithm for converting strings into numbers called?

I've been doing some work in Parsec recently, and for my toy language I wanted multi-based fractional numbers to be expressible. After digging around in Parsec's source a bit, I found their ...
0
votes
1answer
139 views

What are known approaches to graphing algebraic expressions?

I am planning to build an expression parser that will be used to graph algebraic functions ( think TI-83 ) with JavaScript. Functions will take the form of f(x)= Aside from typical operators such ...
4
votes
4answers
2k views

What is the simplest human readable configuration file format?

Current configuration file is as follows: mainwindow.title = 'test' mainwindow.position.x = 100 mainwindow.position.y = 200 mainwindow.button.label = 'apply' mainwindow.button.size.x = 100 ...
0
votes
2answers
284 views

How do I capture information from a website that doesn't provide an API?

Do you know any good tutorials, frameworks, anything that can help me to write code that captures information from a website that don't have a public API, or hasn't been written in a RESTful way? ...
-1
votes
1answer
476 views

Browser parsers vs Stand-alone parsers

Firefox/Chromium have well implemented(?) parsers. Then why are parsers like lxml and other such parsers being developed with so much effort put into it? Aren't they re-inventing the wheel? Or are ...
1
vote
1answer
256 views

Extracting text from various file formats

I want to extract text from various files. I used Apache POI for parsing Microsoft documents. It's working and now I want to parse PDFs and extract text from them. Is there a Java API that I could ...
3
votes
1answer
445 views

How can I test a parser for a bespoke XML schema?

I'm parsing a bespoke XML format into an object graph using .NET 4.0. My parser is using the System.XML namespace internally, I'm then interrogating the relevant properties of XmlNodes to create my ...
9
votes
5answers
1k views

Coming up with tokens for a lexer

I'm writing a parser for a markup language that I have created (writing in python, but that's not really relevant to this question -- in fact if this seems like a bad idea, I'd love a suggestion for a ...
5
votes
2answers
435 views

Parser combinator that looks like BNF

Is it possible to construct a parser combinator library that reads like a BNF grammar? I don't know of any, so I started wondering if there are reasons it's impossible or undesirable to do so. It ...
10
votes
2answers
1k views

What's the simplest example out there to explain the difference between Parse Trees and Abstract Syntax Trees?

To my understanding, a parser creates a parse tree, and then discards it thereafter. However, it can also pop out an abstract syntax tree, which the compiler supposedly makes use of. I'm under the ...
10
votes
5answers
2k views

Are separate parsing and lexing passes good practice with parser combinators?

When I began to use parser combinators my first reaction was a sense of liberation from what felt like an artificial distinction between parsing and lexing. All of a sudden everything was just ...
23
votes
6answers
1k views

What are the arguments against parsing the Cthulhu way?

I have been assigned the task of implementing a Domain Specific Language for a tool that may become quite important for the company. The language is simple but not trivial, it already allows nested ...
14
votes
12answers
4k views

How to write a command interpreter/parser?

Problem: Run commands in the form of a string. command example: /user/files/ list all; equivalent to: /user/files/ ls -la; another one: post tw fb "HOW DO YOU STOP THE TICKLE MONSTER?;" ...
2
votes
1answer
317 views

Idea of an algorithm to detect a website's navigation structure?

Currently I am in the process of developing an importer of any existing, arbitrary (static) HTML website into the upcoming release of our CMS. While the downloading the files is solved successfully, ...