How to start

  • Download sample code submission for competition in zip file.
  • Extract content into a folder and initialise git repository git init.
  • Push it to new repository Instruction for github or Instruction for gitlab.
  • Make sure your repository is public.
  • On repository page click settings -> Webhooks -> Add webhook (it might require confirming password).
  • Set Payload URL https://codalab.coresearch.club/api/competition/{Competition_id}/github_hook, where {Competition_id} should be substituted with an id of a coompetition that you want to participate.
  • Set content type to application/json.
  • Check Just the "push" event.
  • Press Add webhook.
  • Pushing commits to master should trigger submission.
  • On repository page click settings then integrations.
  • Url https://codalab.coresearch.club/api/competition/{Competition_id}/gitlab_hook, where {Competition_id} should be substituted with an id of a coompetition that you want to participate.
  • On Trigger check Push events.
  • Press Add webhook.
  • Pushing commits to master or pressing test button should trigger submission.


    You will need to submit your model predictions in order to receive a score and a leaderboard position in a competition. You are limited to a certain number of submissions per day and per competition. This numbers vary from competition to competition.

  • How can I make a submission?
  • Submitting by using github or gitlab integration is described here. Also you can make submission on competition page in the tab My Submissions. Use this form:

    Insert link then press Submit.

  • How can I look into submissions of other participants?
  • You can find submissions of other participants on competition page in the tab Leaderboard. Leaderboard contains View URL. Click it to follow the github or gitlab repository.

    If you liked any submission you can press Like :)

  • What is public and private leaderboard?
  • The сompetition leaderboard can be private or public. It depends on the competition phase. The public leaderboard provides publicly visible submission scores based on a representative sample of the submitted data. This leaderboard is visible throughout the competition. The private leaderboard, by contrast, tracks model performance on data unseen by participants. The private leaderboard thus has final say on whose models are best, and hence, who the winners and losers of the competition will be. Which subset of data is calculated on the private leaderboard or a submission’s performance on the private leaderboard is not released to users until the competition has been closed.

  • Choose aci worker as Queue manager for your competition in the Codalab and enable "Publish steps files" option
  • Use STEPS env variable in your solution code as a file saving path
  • import os
    steps = os.getenv("STEPS", './')
    def main():
        for x in range(10):
            f = open('{}{}.txt'.format(steps, x), 'w')