Course Project on Exploratory Data Analysis - Discuss and Share Your Work

Problem statement & submission details:

Use this thread to ask questions, share your project, discover interesting projects and give feedback to others.

Guidelines

For the course project, you will pick a real-world dataset of your choice and apply the concepts learned in this course to perform exploratory data analysis. Focus on documentation and presentation - the Jupyter notebook will also serve as a project report, so make sure to include detailed explanations wherever possible using Markdown cells.

Step 1: Select a real-world dataset

  • Find and download an interesting real-world dataset (see the Recommended Datasets section below for ideas).

  • The dataset should contain tabular data (rowsn & columns), preferably in CSV/JSON/XLS or other formats that can be read using Pandas. If it’s not in a compatible format, you may have to write some code to convert it to a desired format.

  • The dataset should contain at least 3 columns and 150 rows of data. You can also combine data from multiple sources to create a large enough dataset.

Step 2: Perform data preparation & cleaning

  • Load the dataset into a data frame using Pandas
  • Explore the number of rows & columns, ranges of values etc.
  • Handle missing, incorrect and invalid data
  • Perform any additional steps (parsing dates, creating additional columns, merging multiple dataset etc.)

Step 3: Perform exploratory Analysis & Visualization

  • Compute the mean, sum, range and other interesting statistics for numeric columns
  • Explore distributions of numeric columns using histograms etc.
  • Explore relationship between columns using scatter plots, bar charts etc.
  • Make a note of interesting insights from the exploratory analysis

Step 4: Ask & answer questions about the data

  • Ask at least 5 interesting questions about your dataset
  • Answer the questions either by computing the results using Numpy/Pandas or by plotting graphs using Matplotlib/Seaborn
  • Create new columns, merge multiple dataset and perform grouping/aggregation wherever necessary
  • Wherever you’re using a library function from Pandas/Numpy/Matplotlib etc. explain briefly what it does

Step 5: Summarize your inferences & write a conclusion

  • Write a summary of what you’ve learned from the analysis
  • Include interesting insights and graphs from previous sections
  • Share ideas for future work on the same topic using other relevant datasets
  • Share links to resources you found useful during your analysis

Step 6: Make a submission & share your work

(Optional) Step 7: Write a blog post

  • A blog post is a great way to present and showcase your work.
  • Sign up on Medium.com to write a blog post for your project.
  • Copy over the explanations from your Jupyter notebook into your blog post, and embed code cells & outputs
  • Check out the Jovian.ml Medium publication for inspiration: https://medium.com/jovianml

Recommended Datasets

Use the following resources for finding interesting datasets:

Example Projects

Refer to these projects for inspiration:

Evaluation Criteria

Your submission will be evaluated using the following criteria:

  • Dataset must contain at least 3 columns and 150 rows of data
  • You must ask and answer at least 5 questions about the dataset
  • Your submission must include at least 5 visualizations (graphs)
  • Your submission must include explanations using markdown cells, apart from the code.
  • Your work must not be plagiarized i.e. copy-pasted from somewhere else.
5 Likes

can i use global terrorism csv?

1 Like

@aakashns
Could you please share dataset for discussion group made max in any lecture or assignment.
please…

1 Like

can we use more than a single dataset(csv file) for the course project??

1 Like

I am trying to use urlretrieve on a link to download a zip of a cvs file from Kaggle. The link when pasted into a browser immediately downloads a zip file on my computer so I know it is correct, but when I use urlretrieve with the url the file I receive is an HTML file of the Kaggle page. I looked on Kaggle forums but still haven’t figured out how to retrieve the cvs file with a url instead of downloading it. Does anyone have any suggestions?

5 Likes

I want to do whatsapp chat analysis. I got this error. What to do?

‘>’ not supported between instances of ‘str’ and ‘int’
what should i do for this error??

@aakashns Please help

!pip install wordcloud

thanks.

What about this? What is the error

Where does the rawToDf function come from? Is it your function or something imported?

If it yours, and you use open function to open this file, then you may try to provide a keyword argument to it: encoding='utf8'.

There’s a possibility that this file uses different encoding. It’s up to you to find which one it is. I’ve just provided the most popular one.

This is rawToDf

So it’s defined inside your notebook.

Add the encoding argument to your open() function with appropriate encoding and it should work.

1 Like

I am copying from this. Kindly check.

1 Like

I dont understant what is meant by that

Doesn’t matter where you copy from, if you use different file, with different encoding, then you have to modify this function. No idea if this will help right away, but it’s the first thing I would do if I encountered such an error.

As I said, add argument to open function. Locate it inside rawToDf function, and add an argument - encoding='utf8'

open(blahblah, blehbleh, encoding='utf8')

2 Likes

image

Is this correct> I am new to python. Kindly bear with me

Also I have uploaded chat to my jupyter notebook . Is that correct?

1 Like

Not here. You’ve pasted the whole code of the rawToDf function. That’s where you should look for open function.

And don’t worry about your abilities of python whisperer, everybody was new at some point :stuck_out_tongue:

You don’t have to if you use it locally, but it’s kinda necessary when using binder. The python seems to be able to open the file, so it’s uploaded correctly, just the language inside doesn’t play well with the default encoding.

image

Is this correct?