Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am using egrep to search our codebase, but at around 300MB of source code that starts to get a little slow. How do you search your codebase?

share|improve this question

closed as not constructive by MichaelT, GlenH7, Jalayn, Yusubov, ChrisF Apr 26 '13 at 22:47

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

3  
Do you really need to search 300MB of sources when searching for something? I think most projects I've worked on had about, well, 2 to 10 MB of source files I think. Perhaps you should move the sources to 'logical' folders/namespaces so you know where a specific method could be. Also separating it into multiple layers could help you out a bit. –  Jan_V Mar 13 '11 at 12:44
5  
How about using ack? –  Job Mar 13 '11 at 16:22
    
What are you searching for? Keyword(s)? File(s)? Class(es)? Visual Studio/Resharper (and I'm willing to bet Eclipse as well) can optimize non-keyword searches. –  Jim G. Apr 24 '12 at 17:27
add comment

12 Answers 12

With Visual Studios "Find in Files".

If your code base gets too big it might be time to split it up in parts. Then you'll know in which part to do your search and it should go faster.

share|improve this answer
    
In the Fine in Files in VS you can save some set of folders with a specific name, so that you can reference them easily in the future. –  earlNameless Mar 13 '11 at 15:24
add comment

Check out ack from http://betterthangrep.com/.

ack is a tool like grep, designed for programmers with large trees of heterogeneous source code.

ack is written purely in Perl, and takes advantage of the power of Perl's regular expressions.

From the site:

Top 10 reasons to use ack instead of grep.

  • It's blazingly fast because it only searches the stuff you want searched.
  • ack is pure Perl, so it runs on Windows just fine.
  • The standalone version uses no non-standard modules, so you can put it in your ~/bin without fear.
  • Searches recursively through directories by default, while ignoring .svn, CVS and other VCS directories. Which would you rather type?

    $ grep pattern $(find . -type f | grep -v '\.svn')
    $ ack pattern
    
  • ack ignores most of the crap you don't want to search

    • VCS directories
    • blib, the Perl build directory
    • backup files like foo~ and #foo#
    • binary files, core dumps, etc
  • Ignoring .svn directories means that ack is faster than grep for searching through trees.
  • Lets you specify file types to search, as in --perl or --nohtml. Which would you rather type?

        $ grep pattern $(find . -name '*.pl' -or -name '*.pm' -or -name '*.pod' | grep -v .svn)
        $ ack --perl pattern
    

    Note that ack's --perl also checks the shebang lines of files without suffixes, which the find command will not.

  • File-filtering capabilities usable without searching with ack -f. This lets you create lists of files of a given type.

    $ ack -f --perl > all-perl-files
    
  • Color highlighting of search results.

  • Uses real Perl regular expressions, not a GNU subset.
  • Allows you to specify output using Perl's special variables. Example:

    ack '(Mr|Mr?s)\. (Smith|Jones)' --output='$&' 
    
  • Many command-line switches are the same as in GNU grep:

    • -w does word-only searching
    • -c shows counts per file of matches
    • -l gives the filename instead of matching lines
    • etc.
  • Command name is 25% fewer characters to type! Save days of free-time! Heck, it's 50% shorter compared to grep -r.

Plus, there's this:

[BinaryMuse ~]: ack --thpppt
_   /|
\'o.O'
=(___)=
   U    ack --thpppt!
share|improve this answer
1  
In Ubuntu there is ack-grep in repositories. –  Tomasz Zielinski Mar 13 '11 at 22:24
1  
Yeah, there are a few different packages for various distros listed on ack's homepage. Personally I just wgot (wgetted?) the standalone version to ~/bin/ack and gave it execute permissions. –  Brandon Tilley Mar 13 '11 at 22:42
1  
add comment

Well, the first way to search is your IDE, whether it's CodeBlocks, Eclispe, IntelliJ or Visual Studio (or whatever) it's bound to have an index of the existing code in the project, and its dependencies.

When we want to peek at someone else's code at work, we have an instance of OpenGrok. It's wickedly fast, really, usually takes less than a second to give you the results of your search. I've only used it though, not configured it, so I don't know about the admin side of things.

share|improve this answer
    
+1 for OpenGrok. –  peakit Mar 13 '11 at 17:02
add comment

AgentRansack is all you need, super fast.

Otherwise RegExBuddy.

share|improve this answer
    
+1 for AgentRansack I use it as well. Has come pretty handy specially when I wanted to find certain details in a bunch of stored procedures! –  MalsR Apr 16 '11 at 20:44
add comment

I've seen places that use a Google Search Appliance for searching code. That's probably overkill, though, even for 300MB of source code. You could set up something a little more cheaply using Apache Solr and a custom web interface. There might even be an off-the-shelf solution along those lines (though I'm not personally aware of any).

Other than that, I also typically use Visual Studio's built-in "Find in Files". Since it has the option to do either a regex-based or sub-string based search, and a sub-string based search is much faster, you might try something like that as well.

share|improve this answer
add comment

Cscope with vim for overall project view. It enables jumping to definition, callers, symbols search or grep for identifiers or to jump to a file. Cscope works very well with C code but fumbles with C++. Exuberant ctags with taglist plugin for vim for per file overview. Exuberant ctags works well with C++ (actually over 40 languages are supported) but does not have support for jumping to callers.

Then there is good old grep for quick and dirty searching within a source tree.

share|improve this answer
add comment

Actually in some ways grep may be your best friend! Grep will handle huge amounts of data and you can chain several greps together to get exactly what you want.

Of course being able to only grep subsets of your code base can be useful too.

share|improve this answer
add comment

My main development environment is Mac OS X and I find Spotlight very helpful when e.g. I'm looking for a snippet of code that I can re-use.

When I'm on Linux I'll typically use a combination of find + grep.

share|improve this answer
1  
+1 SpotLight is really well done. –  user1249 Mar 13 '11 at 18:06
add comment

Two ways:

  • The built-in "Search in source code" on Github which is very fast

  • The "Find in files" function of the IDE if I'm looking for locations that were not pushed to the repository yet

share|improve this answer
add comment

You can get very specific with Mac OSX's Finder

Searching for source code

Or use find, locate and the different flavors of grep

Use fgrep if you can search without regex to get results faster.

share|improve this answer
add comment

Many environments these days will use a sort of indexer to help the user locate where various functions and variables are defined. That's great if you have it, and not so great if you don't. If you are primarily interested in where the routine or variable is defined, there are a number of techniques that can be used to facilitate that search. If however you are looking to determine where they are used and you don't have an indexer, 'grep' (or its derivatives) are probably your best bet.

  1. Function headers. As part of the coding standards at most places I've worked, each routine requires a comment of a specified structure to precede it. Part of this comment includes the routine name followed by a space, which in turn is followed by a special character. Depending upon the company, this special character has typically been either a '-' or a ':'. Not only does this make searching for the routine easy when you know it is in the file you're in, but it also helps get clean and readable output when performing ...

    grep -lrw 'routineName -' *

  2. Routine naming conventions. We tend to name routines using the pattern below. Consequently, based solely on the module abbreviation in the name, we can usually figure out quite quickly where the routine is defined.

    < module abbreviation > < noun > < verb >

  3. Variable naming conventions. We tend to name our variables in a manner similar to the way we name our routines. Again, the module abbreviation helps us determine where the variable is defined.

    < module abbreviation > < description >

Hope this helps.

share|improve this answer
add comment

Funduc Software has a trio of search and replace programs. I just have their Search and Replace Classic, which I use whenever I need to search for something im my entire search tree. Works very well, even with thousands of files.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.