{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Homework #1\n", "Due 11:59 pm EST, **Sunday Feb 13th, 2022**\n", "Email your solutions (both .ipnb and .html files) to:\n", "compscbio@gmail.com.\n", "\n", "### Goal: Ensure that you ...\n", "- have basic, practical knowledge of Python\n", "- are able to install Python packages\n", "- are able to use and create Jupyter notebooks\n", "- can apply this knowledge to explore one aspect of self-renewal\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preamble: functions\n", "In class, we did not cover the syntax for defining your own functions. Let's cover this now. Note that we use the `randint` function from the random module, and so need to import this module. Typically, you should import all of the modules that you will need at the start of your session. So, I am going to put some other import statements here that we won't need until later in the homework. Please install modules, as needed, using either `pip` or anaconda." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [] }, "outputs": [], "source": [ "import random\n", "import nltk\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here is a funky little function. It just prints a string passed to the function a random number of times between 1 and 10." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def myFirstFunction(arg1):\n", " \"\"\"This is just an example; it doesn't do anything useful\"\"\"\n", " times_to_repeat = random.randint(1,10)\n", " for i in range(0,times_to_repeat):\n", " print(arg1)\n", " \n", " return times_to_repeat\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "howdy\n", "howdy\n", "howdy\n", "howdy\n", "howdy\n", "howdy\n", "howdy\n", "howdy\n" ] }, { "data": { "text/plain": [ "8" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "myFirstFunction(\"howdy\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When you create a function, you need to use the `def` keyword, then supply your function name, followed by the names of arguments that need to be passed to the function enclosed in parenthesis, followed by `:`. Then the following indented suite of code is where you define what the function will do. The first line should be text enclosed in three pairs of double quotes. This is what will be returned when `help(yourFunction)` is invoked. Therefore, give some thought to this text and make it a pithy description of your function. If you do not end the function suite with an explicit call to `return`, the the function will by default return `None`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Part 1: Write a function `prose_generator`.\n", "The first thing that you must do for this homeowrk is to define a function that writes a number of English sentences. The user will be asked to specify the number of sentences. The function will also need to take in as input lists of nouns and predicates. The function will construct and print out sentences of the pattern: (noun, predicate), by randomly selecting words from the corresponding lists. Below, we get you started with the skeleton of the function. \n", "\n", "To get full credit for Part 1, your answer must include a function that takes as input the number of sentences to create, and lists of user-supplied nouns and predicates. The lists can be entered either by prompting the user, or by specifying a file(s) that contains the words. If the former, define your examples in your notebook. If the latter, inlcude them as supplemental files with your submission. Your function must randomly select from these lists to make sentences that start with the noun and end with the predicate. Finally, your function must output the specified number of sentences." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "# Part 1: modify the function skeleton below to meet the conditions stated above\n", "\n", "import random\n", "\n", "def prose_generator():\n", " \"\"\" a function that writes [beautiful] prose \"\"\"\n", " # put your code here\n", " \n", " return\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Part 1: Show us an example of running your function here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Part 2: Improve your function\n", "Your function probably does not have much error checking so far. And, it is possible that a random sentence will be generated more than once. I don't know about you, but I find it mildly annoying when people say the same thing repeatedly. So, your second task is to modify your function (or create a new one) so that the following conditions are met:\n", "\n", "(A) That the nouns are actually nouns.\n", "(B) That at least two valid nouns and two predicates are entered \n", "(C) That a sentence is printed at most once. In other words, prevent the function from outputing duplicate sentences.\n", "\n", "To achieve (A), you can compare the user-supplied nouns against the long list of nouns that we show you how to extract from the NLTK resource below. If the noun supplied does not occur in this list, then your function must at the very least, notify the user. To get full credit for Part 2, your new or modified sentence-generator function must do the same things as in Part 1, plus it must meet the conditions (A-C). " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You will need to install the following resources from nltk (see next code cell). For more information about the NLTK resource: https://www.nltk.org/data.html \n", "\n", "The post that we found useful to get a list of lots of Enlgish nouns: https://stackoverflow.com/questions/17753182/getting-a-large-list-of-nouns-or-adjectives-in-python-with-nltk-or-python-mad\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[nltk_data] Downloading package omw-1.4 to\n", "[nltk_data] /Volumes/forest/moved_users/pcahan1/nltk_data...\n", "[nltk_data] Unzipping corpora/omw-1.4.zip.\n", "[nltk_data] Downloading package wordnet to\n", "[nltk_data] /Volumes/forest/moved_users/pcahan1/nltk_data...\n", "[nltk_data] Unzipping corpora/wordnet.zip.\n" ] }, { "data": { "text/plain": [ "True" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nltk.download('omw-1.4')\n", "nltk.download(\"wordnet\")" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "from nltk.corpus import wordnet as wn" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "all_nouns = []\n", "for synset in wn.all_synsets('n'):\n", " all_nouns.extend(synset.lemma_names())\n", " " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "146347" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(all_nouns)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"carrot\" in all_nouns" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\"faba-laba\" in all_nouns" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Part 3: Use Pandas (+ Numpy), and Seaborn (+ Matplotlib)\n", "\n", "Next week (actually this week), we are going to start using Scanpy to analyze scRNAseq data. The Scanpy package is excellent at making it easy to analyze this type of data. But when you are just getting started, this can almost be *too* easy because it hides a lot of important details. The goal of this section is to expose you to some aspects of cleaning and selecting data, and to some basic visualization functions. This will give you a better idea of what is happening under-the-hood with packages such as Scanpy (or Seurat in R).\n", "\n", "In this section, you need to:\n", "\n", "- (A) Load a gene expression data set ([expression table](https://compstemcellbio.s3.amazonaws.com/2022/homework/01/hw1_gex_df.csv); [meta-data](https://compstemcellbio.s3.amazonaws.com/2022/homework/01/hw1_gex_sampTab.csv))\n", "- (B) Remove genes that are not expressed above a selected threshold\n", "- (C) Create boxplot of the expression of one of my favorite genes, Pou5f1, across cell/tissue types\n", "- (D) Identify the top 5 genes in terms of mean expression per cell/tissue type & plot a heatmap of these genes (ordered)\n", "- (E) Compute the similarity of samples & plot a heatmap of them (ordered)\n", "\n", "To do this somewhat efficiently, you might want to look at the following documentation:\n", "\n", "- https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html\n", "- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.max.html\n", "- https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html\n", "- https://seaborn.pydata.org/generated/seaborn.boxplot.html\n", "- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html\n", "- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.mean.html\n", "- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.rank.html\n", "- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_values.html\n", "- https://seaborn.pydata.org/introduction.html\n", "- https://seaborn.pydata.org/generated/seaborn.heatmap.html\n", "\n", "If it sounds like a lot, don't worry it will be fun!\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3A: Load the data\n", "\n", "Download the expression matrix and meta-data from the links above. Use the Pandas `read_csv()` to load these. This will create a Pandas DataFrame. \n", "\n", "This mouse data comes from one of our papers, where we compiled previously published RNA-Seq data for classification purposes. There are 16 cell/tissue types, each represented by approximately 60 or more individual samples. Later, in Step 3C, you should also load the metadata, which indicates the cell/tissue type for each sample.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# 3A: Enter your code for loading the file hw1_df.csv as a Data Frame and displaying part of it." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3B: Filter out genes whose maximum expression < 2\n", "You might consider using the `pd.loc` and `pd.max()` functions" ] }, { "cell_type": "code", "execution_count": 138, "metadata": {}, "outputs": [], "source": [ "# 3B: Enter your code for removing genes and print out the size of the resulting data frame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Limit genes to those that have maximum expression > 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3C: Make a box and whiskers plot of the gene Pou5f1, grouped by cell/tissue type.\n", "To do this you will need to load the meta-data, linked to above. You might want to use the `pd.concat()` and `sns.boxplot()` functions." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# 3C: Show you code for doing what you need to do to ultimatley make the boxplot. \n", "# And, of course, display the boxplot below, too" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3D: Determine the top 5 genes expressed per cell/tissue type\n", "Once you have done so, make a heatmap showing expression of these genes across all samples. To get full credit for this, you need to \n", "\n", "- exclude duplicate genes (i.e. genes that are in the top 5 in more than one cell/tissue type)\n", "- order the samples by cell/tissue type\n", "\n", "You might want to use the Pandas functions `groupby` and `mean` and `sort_values` here, and it may be useful to do some type conversions. Do use `sns.heatmap()` for plotting." ] }, { "cell_type": "code", "execution_count": 140, "metadata": {}, "outputs": [], "source": [ "# 3D: Put your code for this part here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### 3E: Compute the similarity between samples.\n", "Compute the Pearson correaltion coefficient between all pairs of samples. Plot this as a heatmap. Make sure the samples are ordered by cell/tissue type. You might want to use the Pandas `corr()` function. Be careful of what your are computing correlation of.\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# 3E: Put you code for 3E here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Part 4: Modeling Birth and death of CFUs\n", "Recall the self-renewal assay from Siminovitch, Till and McCulloch 1963 (right) \n", "In this assay, nodules were dissected, cells were dispersed, resuspended, and tranplanted into secondary recipients. Then after 10 days (or later depending on the experiment), the number of spleen nodules were counted per recipient. Recall that the number of nodules (or colonies) per spleen was highly heterogeneous. To the authors, this indicated that heterogeneity was \"... a feature either of colony-forming cells or of the process of colony-formation.\" " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " In a follow-up [paper: Till, McCulloch, and Siminovitch, 1963](https://blackboard.jhu.edu/bbcswebdav/pid-11413450-dt-content-rid-113157130_2/xid-113157130_2), this trio proposed that the heterotengeity could be due to inherent stochasticity in each colony-forming cell. In other words, a colony forming cell \"... either give[s] rise to progeny like itself (\\\"birth\\\"), or [is] removed in some way (\\\"death\\\"), and these two events occur in a random fashion.\" This is often referred to as a \"Birth-and-death process\". They reasoned that a cell will give birth to another colony forming cell with Probability = p2 or to a differentiated cells with Probability = p0 (see panel at left). They further reasoned that if this model is correct, then simulations using the model must be aligned with their empircal observations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The goal of Part 4 is for you to replicate some of what the authors did to test this reasoning:\n", "\n", "> A test of this assumption was made by the use of the Monte Carlo method. In this method, the choice between occurrence or nonoccurrence of a random event is made by drawing a number from a table of random digits. Consider as an example the case in which the birth and death probabilities, p2 and p0, are arbitrarily set equal to 0.6 and 0.4, respectively. Let the six digits, 0, 1, 2, 3, 4, and 5 signify a \"birth\" and the four digits 6, 7, 8, and 9 signify a \"death.\" If the first random number to be drawn was a 5, then the model would show a birth, or an increase from one cell to two cells. Each subsequent choice of a number would determine the fate of a cell in a similar manner." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Part 4A: Write a function to simulate the number of colony forming units.\n", "Your task is to precisley mimic what the authors state above: write a function that takes as input:\n", "\n", "- the number of time steps, or generations\n", "- P2 = probability of birth, resulting in two colony forming units \n", "\n", "and that outputs the number of CFUs remaining. Note that a death event results in a loss of one CFU, whereas a birth yields two CFUs. You may consider using the Numpy function `random.uniform()` here." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# 4A: Write your function definition here" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# 4A: Show us an example of how it works here" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Part 4B: Compare *in silico* experiments to empirical data\n", "Use your function to determine how values of P2 influence the distribution of CFUs, and how this compares to real data.\n", "To do this, you will need to call your function many times. For this homework, let's stay close to the original paper (n=52 recipient spleens).\n", "You will also need the original [data](https://compstemcellbio.s3.amazonaws.com/2022/homework/01/hw1_cfu_data.csv), which you can load with the usual `pd.read_csv()` function.\n", "We recommend testing 3 - 5 values for P2. As you will see from the original paper, the simulated and experimentally derived data were compared by examining cumulative distribution functions. You can readily plot these using the Seaborn `ecdfplot()` function.\n", "\n", "Bonus points: Compare these distrubitions to the Gamma distribution mentioned in the paper." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# 4B: Show your code and figure(s) here." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 4 }