Case (I) python warm-up
Project 3: Crawling the university ranking of a province
Enter the name of a province and crawl the data from "SoftTech China's Best Universities Ranking 2020" (http://www.zuihaodaxue.cn/zuihaodaxuepaiming2020.html) developed by Shanghai Jiao Tong University to output the province's 2020 University rankings for that province in 2020.
Input: Guangdong
Output.
import requests
from bs4 import BeautifulSoup
import bs4
def getHTMLText(url):
try:
r = requests.get(url, timeout=30)
r.raise_for_status()
r.encoding = r.apparent_encoding
return r.text
except:
return ""
def fillUnivList(ulist, html):
soup = BeautifulSoup(html, "html.parser")
for tr in soup.find('tbody').children:
if isinstance(tr, bs4.element.Tag):
tds = tr('td')
ulist.append([tds[0].string, tds[1].string, tds[2].string])
def printUnivList(ulist, num, place):
tplt = "{0:^10}\t{1:{3}^10}\t{2:^10}"
print("{:^10}\t{:^6}\t{:^10}".format("排名", "学校名称", "省市"))
for i in range(num):
u = ulist[i]
if u[2] == place:
print(tplt.format(u[0], u[1], u[2], chr(12288)))
# print("{:^10}\t{:^6}\t{:^10}".format(u[0], u[1], u[2]))
else:
continue
def main():
uinfo = []
url = 'http://www.zuihaodaxue.cn/zuihaodaxuepaiming2019.html'
html = getHTMLText(url)
fillUnivList(uinfo, html)
printUnivList(uinfo, 549, "广东") # 20 univs
main()
Results of the run.
From the data items returned by the functions listed in the table above, the interface function returns a very rich variety of data that can be used to reflect the value of the company's investment in a number of combinations of data items. For example, the return parameters from the three types of data listed in the table above select the net interest rate, return on net assets, earnings per share growth rate, net profit growth rate, earnings per share growth rate, the ratio of net operating cash flow to net profit and cash flow ratio as key evaluation indicators, and the comprehensive use of the value of these seven indicators to reflect the quality of the company's fundamentals. Generally, the higher the value of these indicators, the higher the fundamental quality of the company and the greater the investment value of the company's shares. The following program demonstrates a method of using financial indicator data to screen for quality company stocks. 
如果在If you have saved the financial data file before running the program, you can read the financial data directly from the local data file, thus avoiding the need to download the same data every time you run the program.
The disclosure time for financial reports of A-share listed companies in China is stipulated as follows: the first quarterly report is disclosed by 30 April, the half-yearly report by 31 August, the third quarterly report by 30 October, and the annual report is disclosed by 30 April of the following year. The program uses the date function of the datatime package to determine the available financial report data.
The size of the dataset is 3823x22 and each row is the basic data for one stock. See Tushare's website for field details. See the 
As can be seen from the above statistics, the more economically developed and dynamic the region, the greater the number of listed companies. The reader can also perform similar statistics by industry. The timeToMarket field in the data box represents the date of listing and is an integer in the format of "20190315". We can extract the year from it to count the number of shares issued each year. 
从图中It can be seen that several highs in the year of IPO issuance correspond to several bull market times in the domestic stock market, with the number of issuances falling to a low during bear markets. The following calculates the market's average price-to-earnings ratio, pe, which is an important parameter in measuring stock market valuation. 
Looking at the dataset reveals that the pe of loss-making stocks in the dataset is 0. Therefore, the removal of loss-making stocks is considered. 
The pe above is a simple arithmetic average, a weighted pe with market capitalisation as the weighting may be a more accurate reflection of market conditions. As the total market capitalisation and stock unit prices are not available in the downloaded dataset, the total market capitalisation can only be extrapolated from the available fields. It is also common in data processing to calculate new column values from the values of certain columns. Here the total market capitalisation is extrapolated on the basis of
Unit price of stock = 4esp (earnings per share) pe (price-to-earnings ratio)
Total market capitalisation = share unit price *totals total equity (RMB billion)
The earnings per share esp in the dataset is for a single quarter, so multiply the full year earnings by 4.
The above calculation reflects the market-weighted pe after a particular quarterly report and the result differs from the true market value. This is because stocks have different returns each quarter and therefore you cannot simply calculate the full year return on a "4*single quarter return" basis.
The data returned by Tushare's built-in functions are all of the DataFrame type of Pandas, so it is easier to manipulate the data returned by these functions using the manipulation tools provided by Pandas, NumPy, Matplotlib and other packages.
Return Value Description:
(3) Use the command pip install tushare to install the Tushare package locally.
Return value description.
Other interest rate pro interface functions.
(1) pro.shibor_quote():Shibor quoted data
(2) pro.shibor_lpr():LPR lending base rate
(3) pro.libor():LIBOR
(4) pro.Hibor():Hong Kong Interbank Offered Rate
Return value description.
There are some structural differences between the quotation data obtained with the pro interface function pro.daily() and the quotation data obtained with the get_hist_data() function. The return data from the pro.daily() function uses the default index, with the date of the transaction as a field (column) data item, whereas the return data from the get_hist_data() function uses the date of the transaction as an index.
Given that the structure of the data returned by the different interface functions differs to a greater or lesser extent, it is important to design programs that process this data in such a way that the appropriate processing method is chosen based on the structure of the data returned, or that the data items returned are modified as necessary to meet the formatting requirements of other data processing statements.
Tushare also provides real-time data on stock trading quotes, i.e. data on the prices of stocks that are being traded on that day. For example, the real time stock trading data obtained through the get_realtime_quotes() function can include information such as the list of quotes and transaction prices of the stock at the current moment, the five buy quotes and five sell prices and other data items, with a total of more than 30 items of information. The operation process is as follows. 
Partial return value description.
The following two tables list the input and output parameters of the Tushare platform pro version interface function index_basic(). 
It looks like the results have been run, but the long string in front of it is really uncomfortable to watch. This, coupled with the fact that ==Process finished with exit code 1== makes me even more convinced that there is something fishy going on here.
As you can see, the headache-inducing red error message is gone, and ==Process finished with exit code 0==
The problem seems to have been solved, but we know that this is only superficial and that the problem has not been solved, but we have chosen not to display it. The cure is not the cause!
After removing the blank line at the end, run the program again:
Now that's perfect!
Comfortable =。=
A variety of different styles of images will be produced in the tmp directory.