Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with "streams of text/topics" rather than simple words etc.) and I am thinking of using the Wikipedia XML dumps for the purpose.

I guess I need to learn parsing an XML and "some other things" related to NLP and probably Machine Learning, but I am no way sure about anything involved herein after the XML parsing.

  • Is the starting step: XML dump parsing into text a good idea/step? Any alternatives?
  • What would be the steps involved after parsing XML into text to create a functional Semantic Network?
  • What are the things/concepts I should learn in order to do them?
  • I am not directly asking for book recommendations, but if you have read a book/article that teaches any thing related/helpful, please mention them. This may include a refernce to already existing implementations regarding the subject.

Please correct me if I was wrong somewhere. Thanks!

EDIT: The final product should be like a complete Semantic Network (like Conceptnet or Cyc etc.), So I can't use things like Semantic Mediawiki.

(On a second thought, it seems like I should have asked this question on linguistics and not here... )

share|improve this question
    
what is wordnet?? –  nischayn22 Jun 25 '12 at 19:40
    
@nischayn22: Wordnet is an example of a Semantic Network. Simply putting, it links different words using relations (like synonyms etc.) to form a network of lexical relations between the vocabulary, and can be used both as dictionary, thesaurus and also for various AI applications. You can read about it here: wordnet.princeton.edu and here: en.wikipedia.org/wiki/WordNet –  Forbidden Overseer Jun 26 '12 at 14:34

2 Answers 2

up vote 2 down vote accepted

Wikipedia is great for human processing, not so much for machine processing. Have a look at http://wiki.dbpedia.org to get the data extracted from Wikipedia in a machine processable format. Their data set uses OWL as a knowledge representation to provide semantic context to the Wikipedia data set through RDF (XML based) metadata.

Once you have the data set, you will need to be able to run queries against your data. SPARQL is particularly useful in querying data exposed in this format. Consider how you define a stream of data, how you implement your data structures, and how you implement your algorithms that query your data to connect it all.

You can check dbpedia tagged questions for additional guidance.

http://answers.semanticweb.com/tags/dbpedia/?sort=newest

http://stackoverflow.com/questions/tagged/dbpedia

share|improve this answer
    
Your answer made me re-think of Dbpedia as an option... I was reluctant to use Dbpedia for the purpose, but since you mentioned it, it does seem like a better option than Wikipedia. I will take a look at DBpedia like you said... –  Forbidden Overseer Jun 28 '12 at 21:52

Since you mentioned semantic network, you might use semantic mediawiki, I think the process will be to download the database dumps of Wikipedia and adding semantic properties wherever needed. Also are you aware of the Wikidata project??

share|improve this answer
    
I knew about Semantic Mediawiki before, but I don't think using it for this purpose would be a good idea. The main purpose here is to create a Semantic network based on Wikipedia... I think it was my mistake that I gave the example of Wordnet over there, but I guess I should have mentioned something else. I will edit the question a little now... –  Forbidden Overseer Jun 26 '12 at 14:25
    
I really appreciate that you wanted to help... Answering the question you asked above: No, I wasn't aware of the Wikidata project! But, that doesn't seem to help me in this situation dude... –  Forbidden Overseer Jun 28 '12 at 21:54
    
@ForbiddenOverseer for more help you really have to ask the mediawiki guys on irc –  nischayn22 Jun 29 '12 at 5:38

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.