Press "Enter" to skip to content

Generating Description Files for Code-Eval Challanges with Perl

I’ve put my solutions for the code-eval challenges on github, but as I was working
on them I only saved the solutions. Not to worry, this is a great opportunity to learn
more about Perl.
The URLs for the challenges follow the format of
https://www.codeeval.com/open_challenges/ with some number at the end.
Following ScriptSocket’s excellent tutorial on fetching websites and converting
them to text we can scrape the web pages and search them for the relevant
information. Ettiquette demands we sleep for five seconds between requests and it’s
much more polite to output the data to some file instead of keep downloading
everything.

#!/usr/bin/perl
use LWP::Simple;
use HTML::TreeBuilder;
use HTML::FormatText;

foreach(1..240){
  my $URL = get("https://www.codeeval.com/open_challenges/$_/");
  print "Fetching Challenge $_ \n";
  my $Format = HTML::FormatText->new;
  my $TreeBuilder = HTML::TreeBuilder->new;
  $TreeBuilder->parse($URL);
  my $Parsed = $Format->format($TreeBuilder);
  $code_eval_data .= $Parsed;
  $sleep(5);
}
open($OUTFILE, ">", "code_eval_data");
print $OUTFILE $code_eval__data;

Now we have the data we can parse it. Generally the descriptions start with the
challenge name and end with “login to submit solution”. On my system the
solutions are stored in folders that share the challenge name. The basic premise is,
for each folder search the data for a string that starts with the name of the folder
and ends with “login to submit solution” and write it to a file in that folder called
description. Failing that, print out the challenge name so it can be added manually /
the script can be updated.

use File::Slurp;
$file = "code_eval_raw";
my $text = read_file($file);
foreach(grep(/\.\/\w+\//, `find . -type d`)){
  chomp $_;
  open($description, ">", "$_/Description");
  $_ =~ /\.\/\w+\/([\w-]+)/;
  $challenge_name = $1;
  $challenge_name =~ s/-/ /g;
  if($text =~ /$challenge_name/i){
    $text =~ /$challenge_name(.*?)(Login to submit solution)/si;
    print $description $1;
  }
  close($description);
  else{
    print "Couldn't do $challenge_name\n";
  }
}

 

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *