Not your usual #science #blog
When making projects with Python, watch out for your file directory design!
web scraping websites with javascript - it gets complicated

Web scraping websites that uses javascript to create a html page gets complicated. A fellow alumni alluded to that in a recent Facebook conversation we had: “It gets more complicated if you have to deal with AJAX sites, but unfortunately I’ve dealt with all this nonsense before. “

Links that I found useful:
http://stackoverflow.com/questions/4720342/python-processing-javascript-with-urllib2
First I searched on google about things on webscraping javascript which led me to this stackexchange question:
http://stackoverflow.com/questions/8049520/web-scraping-javascript-page-with-python
which led me to this link: (shoutouts to bpgergo)
http://www.packtpub.com/article/web-scraping-with-python-part-2
http://blog.databigbang.com/web-scraping-ajax-and-javascript-sites/

Stackexchange is great because I could even look up whether going with one package is better than the other:
http://stackoverflow.com/questions/719624/selenium-vs-windmill

I found that the documentation for Selenium on its own website isn’t straightforward for installation onto the Mac OS X, using the Terminal. This gentlemen, Damien, had a great explanation on his website:
http://damien.co/resources/how-to-install-selenium-2-mac-os-x-python-7391

Using requests python library to get a website, but you get a 403 Forbidden; throw in your browser header!

For this case, I’m just going to use the website seekingalpha.com as an example.

So I find myself trying to use requests to get a website but I get a
403 Forbidden

David Marcus, fellow alumni, told me:

"You may have to pass the same headers that your browser does in order to your query look more legitimate.
This is easy with the requests library.”

So I tried this as he told me:

import requests
headers = { ‘User-Agent’: ‘Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36’, }
r = requests.get(‘http://seekingalpha.com/symbol/PXD/transcripts’, headers=headers)
print r.content

Works.

Algebraic Geometry - sheaves

I was reading about Fiber Bundles from Jeff Lee. Which led to sheaves from Algebraic Geometry. Which leads me back to rings and modules. So I’m working on rings and modules (understanding them) for the next few hours.

isometries:

alexyar:

isometries said: wait what the fuck do they not use epsilon delta here???

heh, aren’t you at Pitt? ‘Cause remember, we’re talking about the department where some instructors don’t teach Mean Value Theorem because it’s “too hard” and “too theoretic” and the problems on it “are…

I see. Mostly this just made me sad, but wow did I learn a lot. I took all of calc in high school, so I never had to do any of that here, aside from helping the occasional friend-in-calculus (but now that I think of it, all those limit problems and not a greek letter to be seen…). I just find it so strange that epsilon-delta limits or even Mean Value Theorem aren’t taught in calc 1 here when I actually did learn them in high school, and I suppose I naively assumed that introductory calculus is the same everywhere. I didn’t go to any especially fancy high school (public school in suburbia), but we definitely tackled calculus with more rigor and depth than it seems like Pitt does. And I’ve only been here a year, but for all the courses I’ve taken, I feel the department’s been great. It’s like there’s an entirely different attitude taken by the department toward calculus than there is toward the rest of math. But I agree with you completely that these things need to be fixed at the high school level, and I take it that the department does what it sees as best to accommodate these students who really don’t have the proper algebra and theoretical reasoning backgrounds.

So much for having high standards for our US students for math skills.

1. The problem is NOT you guys, the instructors, because you guys, the instructors, know how to communicate and have mastered this stuff.
2. It’s already bad enough that standards have been lowered, but I can’t believe students can’t do fractions. What are they doing in college?
3. The problem is NOT lack of money nor education resources. There are (amazingly) plenty of lower level, basic, introductory math videos on the internet and YouTube. I am amazed at what I can learn on the internet. Yes, there are a lot of silly videos on Vine, but thank goodness the internet is “big” enough to also accommodate enough videos for a lifetime of learning.
4. The problem is the US students themselves. A lot of them don’t belong in a college classroom.

madewithpaper:

What’s Your Adventure?
Made With Paper by elevenyellowbeagles

Love it

madewithpaper:

What’s Your Adventure?

Made With Paper by elevenyellowbeagles

Love it

I was in Most today.

I was in Most today. If you don’t know where or what Most, Czech Republic is, look it up. Essentially, Communism, literally, ran it into the ground - it’s been so strip-mined, coal-mined, and uranium-mined that it’s an environmental and economic disaster zone. It’s beautiful countryside - other than the 6 nuclear power plants, coal smoke stacks, failed heavy industry. Rails for trams and trains are rusting. Everything in Most is 25 years back - it’s the city that Communism had left. The fashion, cars, even technology is 25 years back. Immediately as I stepped out in Most, you could feel the desperation. All it consisted of were blocks and blocks and blocks of Communist era apartment buildings - the kind of apartment buildings with paper thin walls - great for spying on others.

I was waiting for the bus in Most and three children walked up to me. They didn’t have socks, it was obvious they were wearing hand me downs. They asked me what the time was in Czech. I really couldn’t make out what they said. The oldest girl, probably no older than 12 years, asked me in English the time. Then she asked me where I was from. “I’m from Los Angeles! California!” I replied. The three of them lit of with glee and joy with the biggest smiles on their faces. They asked me what I was doing in the Czech Republic and that they wanted to speak more English but only knew a little. They asked me what LA was like. They asked me what America was like.

I asked them where they lived. “Children place.” They lived in an orphanage. They were orphans.

Just my luck, it thundered and poured heavily as we were all still waiting for the bus to arrive. We scrambled away for shelter, but the children kept looking back at me, with the biggest smile on their faces. It was as if I was another being from another planet. You know that internet meme that’s caught on recently of that criminal with the model looks and the bright, beautiful blue eyes and the chiseled face, who’s now getting $30k from a kickstarter to help bail him out? The youngest boy, he honestly looks like a 6 year old version of that guy. He kept smiling, looking back with earnest curiosity and sincerity, as I caught my bus out of Most.

And that right there was what being, ideally, an American meant. We’re not Americans because of our tyrannical military might, we’re not Americans because of our history, because goodness the apartment building I was living in is older than America. What makes us American is our values - hope for a better life, the opportunity for a better life, the freedom to work hard for a better life. What makes us American is what we represent - that no matter what color you are or how poor you started, you could make a better life for yourself anew. Our first world problems are these kids’ dream paradise.

Happy Fourth of July ya’ll. #Merica

Math humor.

Honestly, I was looking up how to compute stabilizers, normalizers and all that fun stuff from abstract algebra for Dihedral groups and this was the first thing that came up on the Sage Math documentation: JOKE: Q: What’s hot, chunky, and acts on a polygon? A: Dihedral soup. Renteln, P. and Dundes, A. “Foolproof: A Sampling of Mathematical Folk Humor.” Notices Amer. Math. Soc. 52, 24-34, 2005.

In 1958, Stephen Smale stunned the mathematical community and here’s why………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………..

He proved it was possible to turn a sphere inside out without introducing any creases.

sometimes my code doesn’t work, then i remove something completely unrelated, and then it works. =| cf. autoincrement for sqlite in SQLAlchemy

If you want to know what tables are in SQLAlchemy already in a Base

If you want to know what tables are in SQLAlchemy already in a Base,

import sqlalchemy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()

for t in Base.metadata.sorted_tables:
print t, t.name

Hi, if you use Pythonista, then in the latest version 1.5, the Open in feature has been disabled. The creator @olemoritz has a very lucid, well-defended, and reasonable explanation for this: http://olemoritz.net/pythonista-15-whats-new-and-whats-missing.html and I agree with his analysis entirely. Nevertheless, for a complete novice and beginner like me, with developing or programming in iOS, here is a very simple work around that I could appreciate would not be obvious to someone who is completely new to Python, Pythonista, and iOS or even an iPad.

The screenshots above walk you entirely through what to do. Essentially, email yourself your Python script. Copy the entire thing and paste it into your desired directory in Pythonista.

Have fun!

I sit in bed, wondering if the universe was a closed, oriented sphere. I would travel the entire length of the universe, only to come back, standing right behind her, with her right hand clutching her clutch, as she had always been. #MadeWithPaper (fiftythree.com)

I sit in bed, wondering if the universe was a closed, oriented sphere. I would travel the entire length of the universe, only to come back, standing right behind her, with her right hand clutching her clutch, as she had always been. #MadeWithPaper (fiftythree.com)

Made With Paper category theory, functor, morphism

Made With Paper category theory, functor, morphism