Analyzing (un)structured data to convert it into a structured, normalized format.

learn more… | top users | synonyms (1)

3
votes
5answers
694 views

What would happen if you defined your system's CSV delimiter as being a quotation mark?

Title says it all. If the CSV's system delimiter was " (as opposed to a comma or pipe or other common alternatives), how would anything deal with it? The crux of the matter is of course that by ...
3
votes
1answer
127 views

Object oriented parser model review

First time I make use of Code Review. At the moment I'm creating an Android application and the company I'm developing it for, has a specific file format that can be used to give text based data ...
3
votes
1answer
691 views

Extracting color profile information from JPEG files

I'm trying to look up info about reading JPEG's color profile info and to my surprise there's very little open specific how-to information on that regard, but rather lots of general explanation on ...
3
votes
1answer
157 views

How to support mixed grammars?

I am currently writing a grammar for a programming language (PowerBuilder) and I am also collecting documentation about parsing and compilers creation. Now I would like to add in my grammar (that ...
3
votes
1answer
146 views

Chosing a parser for a code beautifier

I'm in the planning stage of making a code beautifier (similar to AStyle or Uncrustify) - originally I was going to just contribute to one of those projects, but reviewing their source led me to the ...
3
votes
2answers
956 views

How to get lookahead symbol when constructing LR(1) NFA for parser?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.292 in the 2nd edition) about how to construct an LR(1) parser, and I am at the stage of building the initial ...
3
votes
3answers
347 views

design for a parser to handle very large files

I have written a program which records protocol messages between an application and a hardware device which matches each application request with each hardware response. This is so that I can later ...
3
votes
1answer
146 views

Performance issues when parsing code [closed]

I was wondering about performance issues when parsing a source file that is being edited by the user (for example, when you need to give a syntax highlight). I think that the simplest approach is to ...
3
votes
2answers
125 views

Sentence Tree v/s Words List

I was recently tasked with building a Name Entity Recognizer as part of a project. The objective was to parse a given sentence and come up with all the possible combinations of the entities. One ...
3
votes
1answer
187 views

What does it mean that StAX parsers aren't complete or correct?

I've been browsing the net and found theses two slides on site of XOM (XML object model): slide1 slide2 Where they refer to SAX(push) and StAX(pull) parsers. So I'm wondering, what does it mean ...
3
votes
1answer
445 views

How can I test a parser for a bespoke XML schema?

I'm parsing a bespoke XML format into an object graph using .NET 4.0. My parser is using the System.XML namespace internally, I'm then interrogating the relevant properties of XmlNodes to create my ...
3
votes
2answers
6k views

First and Follow Sets for a Grammar [closed]

I'm studying for a Compiler Construction module I'm doing and I have a sample question as follows: Calculate the FIRST and FOLLOW sets for the following grammar.. S -> uBDz B -> Bv B -> w D ...
2
votes
2answers
150 views

parsing terminology: comments+whitespaces vs actual code

In languages like c/c++ spacing and comments are ignored and only actual code gets into compiler. I'm interested if there is accepted way of naming these two things? comments & spacing ...
2
votes
3answers
2k views

c++ write own xml parser vs using tinyxml

I am currently in a task to generate an XML file for an srt text file containing timestamps and corresponding text. To generate an exe file which accepts file name input and outputs the relevant XML ...
2
votes
3answers
1k views

What programming language is most suitable for handling unstructured data?

I'm trying to automate the application of metadata to huge amount of text, but I'm not sure what language would make this task easier (if there is one). What programming language is most suitable ...
2
votes
4answers
206 views

The right place to embed a parser : Server or the Client?

I'm writing a database management software. It has a client-server architecture where the server has the piece of code to store and retrieve. The client has to get the input from the user, connects to ...
2
votes
1answer
197 views

REST efficiency on count and sum methods

For example, if I have a Customer Transactions Table and I create a REST for it. The fields are: date, description, and total_amount. I am trying to figure out which one is much more efficient when ...
2
votes
1answer
535 views

Any ASN.1 parsers written completely in managed code & safe from buffer overflows

ASN.1 is a data format used on HTTPS certificates, and various other critical pieces of infrastructure. I need to analyze some potentially hostile ASN.1 data, and want to ensure my code is safe from ...
2
votes
4answers
316 views

Is it possible to create a single tokenizer to parse this?

This extends off this other Q&A thread, but is going into details that are out of scope from the original question. I am generating a parser that is to parse a context-sensitive grammar which can ...
2
votes
1answer
317 views

Idea of an algorithm to detect a website's navigation structure?

Currently I am in the process of developing an importer of any existing, arbitrary (static) HTML website into the upcoming release of our CMS. While the downloading the files is solved successfully, ...
2
votes
3answers
1k views

Writing a CSS parser in C#. What do you think is the best strategy?

I'm in the middle of writing a CSS parser in C#. I'm well under way, but I also have those times where I wonder if I'm taking the best approach. The things I've considered are: Feed the CSS grammar ...
2
votes
1answer
132 views

content fingerprint algorithm

I have a database containing just simple URLs. It is as simple as it sounds for now and URL can link to a website or a document(i.e. anything parseable to text). Now I have a simple code which ...
2
votes
2answers
1k views

An ideal way to decode JSON documents in C?

Assuming I have an API to consume that uses JSON as a data transmission method, what is an ideal way to decode the JSON returned by each API resource? For example, in Java I'd create a class for each ...
2
votes
1answer
501 views

How to add precedence to LALR parser like in YACC?

Please note, I am asking about writing LALR parser, not writing rules for LALR parser. What I need is... ...to mimic YACC precedence definitions. I don't know how it is implemented, and below I ...
2
votes
1answer
219 views

Best way to parse optional grammar rules?

I started to write a hand written parser just for fun. For simple rules it is working, but I have problem with the optional grammar rules. I marked the optional rules with a question mark. Here is a ...
2
votes
1answer
283 views

How to choose a proper parser generator for PHP

Some programmers avoid regexes in some situations (see this popular @nickf comment), perhaps using a parsing framework such as Lex/Yacc. Others prefer to stay within PHP, perhaps using regular ...
2
votes
1answer
194 views

Practical reference for learning about graph reduction

Are there any practical references (with actual examples) for getting started implementing a small, lazy functional programming language with graph reduction? A reference that included the lexing and ...
2
votes
2answers
3k views

Fastest C++ XML parsing library

I have thousands of .xml files from size 1MB-45MB (no DTDs). I need to parse and further manipulate these XML files before generating separate .xml files with the results of my regex. What the ...
2
votes
1answer
623 views

Using python to parse log files? [closed]

My first useful projects as a programmer has been python scripts that parse out relevant information from log files and do some analysis. I've bumped around and found my way to some functional ...
2
votes
2answers
136 views

Resources for writing a parser combinator library [closed]

I often use parser combinator libraries, but I've never written one. What are good resources for getting started? In case it matters, I'm using Julia, a functional (but not lazy) language.
2
votes
1answer
268 views

LL(∞) and left-recursion [closed]

I want to understand the relation between LL/LR grammars and the left-recursion problem (for any question I know parcially the answer, but I ask them as I don't know nothing, because I am a little ...
1
vote
5answers
742 views

I need to develop a parser. Can I use Lex and Yacc for the purpose?

I need to extract very particular data from log files(of different types and formats). Since I am a recent college passout; my mind ran to using Lex and Yacc for the purpose. Now I have the following ...
1
vote
2answers
137 views

Parser and interpreter knowledge as a way to gauge programmer ability [closed]

This is only anecdotal evidence but from my past encounters with programmers at various workplaces the programmers that understand the fundamentals of parsing and interpreting seem to be overall ...
1
vote
1answer
142 views

Grammar: start: (a b)? a c; Input: a d. Which error correct at position 2? 1. expected “b”, “c”. OR expected “c”

I am writing a parser (a parser generator for PEGs) and am trying to figure out how to report errors. Grammar: rule: (a b)? a c ; Input: a d Question: Which error message correct at position 2 ...
1
vote
1answer
431 views

Picture Parsing [closed]

If I open a picture file, lets say with an PNG extension, I will see bunch of code. Now let say I want to get some information from the picture mechanically. So the question here is what is the first ...
1
vote
2answers
791 views

How to create a Semantic Network like wordnet based on Wikipedia?

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with ...
1
vote
1answer
256 views

Extracting text from various file formats

I want to extract text from various files. I used Apache POI for parsing Microsoft documents. It's working and now I want to parse PDFs and extract text from them. Is there a Java API that I could ...
1
vote
1answer
229 views

PHP Simplest XML tokenizer for string processing?

I need to do some text processing tasks with short XML fragments. The section "details" shows an example... My solution is to use a tokenizer based on regular expresions, but it is not elegant and not ...
1
vote
2answers
137 views

How are line/column position data dealt with in parser combinator libraries?

I'm building a parser using a parser combinator library. I need to keep track of where AST nodes started and ended in the textual input -- line and column numbers. How is this problem approached ...
1
vote
1answer
63 views

About AST construction in LL1 non recursive parser

I have implemented a LL1 parser in a non recursive approach with a explicit stack. The following algorithm is from the Dragon Book: set zp to point to the first symbol of w; set X to the top stack ...
1
vote
2answers
285 views

How would my custom language be categorized?

I'm developing my own scripting language to solve some unique challenges for a project. The language takes source code and converts the contents into tokens, and then a command factory is used to ...
1
vote
2answers
83 views

Creating documents from snippets

From time to time I create whole documents (e.g. XML or proprietary) programmatically. If it easy enough, I just write something like stringbuilder.append("<blah>"); ...
1
vote
2answers
265 views

Should I use a formal grammar for my interpreted scripting language

I have a scripting engine I just published as an open source project. It's been sitting on my harddrive waiting for about a year. My engine of course isn't complete in any way, but it does work for ...
1
vote
2answers
150 views

Creating a text input simplifaction tool

I have been working on a project for several months now to write a web-based tool that will help me at work. I work at a call center and the CRM software that is used is wretched, we basically have to ...
1
vote
2answers
206 views

How to extract operators from the grammar productions for conflict resolution in LALR parser?

Is there some standardized or widely accepted algorithm for picking up operators in shift/reduce conflicts in LALR parser? The question is naive, my problem is not with implementing my solution, but ...
1
vote
2answers
122 views

Extracting useful information from free text

We filter and analyse seats for events. Apparently writing a domain query language for the floor people isn't an option. I'm using C# 4.0 & .NET 4.0, and have relatively free reign to use ...
1
vote
0answers
45 views

Compound container for storing output from a parser

Consider a hierarchical structure like this directory [ [item_0;item_1;...;item_n] [item_0;item_1;inner_directory[...];item_m] other_directory [...] ] ...
1
vote
0answers
119 views

Implementing semantic actions in table driven predictive parsing

When doing table driven predictive parsing on an LL(1) grammar, (as explained in good detail here for example), how can we augment the algorithm to allow processing of semantic actions while doing the ...
1
vote
1answer
164 views

How to read reduce/shift conflicts in LR(1) DFA?

I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when ...
1
vote
1answer
206 views

Does JAXP natively parse HTML?

So, I whip up a quick test case in Java 7 to grab a couple of elements from random URIs, and see if the built-in parsing stuff will do what I need. Here's the basic setup (with exception handling etc ...