Month: November 2023

I used a statistical method called regression modeling to take a closer look at two important things: the stats about Logan International Airport and how full hotels were. These factors played the lead roles in our study, helping us understand how they influence economic indicators.

By digging into the numbers, I didn’t just learn about the direct impact of transportation and hotels on the economy. I also uncovered hidden connections that, when pieced together, paint a complete picture of how these factors shape the larger economic scenario. This deep dive into the data gave us valuable insights and helped me grasp the complex forces that drive economic trends. It was a reminder of how crucial it is to consider various factors when thoroughly analyzing data.

 

 

Over the past few days,  I delved into the realm of advanced statistical analyses, with a primary focus on regression modeling. This sophisticated technique empowered us to systematically quantify relationships between crucial variables, injecting a quantitative dimension into our previously qualitative observations. This analytical step marked a pivotal moment as we sought to unravel the intricate web of connections within our data.

OLS Regression Results                            
==============================================================================
Dep. Variable:      med_housing_price   R-squared:                       0.691
Model:                            OLS   Adj. R-squared:                  0.683
Method:                 Least Squares   F-statistic:                     90.56
Date:                Thu, 30 Nov 2023   Prob (F-statistic):           2.21e-21
Time:                        21:09:15   Log-Likelihood:                -1103.1
No. Observations:                  84   AIC:                             2212.
Df Residuals:                      81   BIC:                             2219.
Df Model:                           2                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
const       1.931e+05   5.58e+05      0.346      0.730   -9.18e+05     1.3e+06
unemp_rate   1.23e+07   2.19e+06      5.625      0.000    7.95e+06    1.66e+07
total_jobs    -1.4689      1.347     -1.091      0.279      -4.149       1.211
==============================================================================
Omnibus:                        3.255   Durbin-Watson:                   0.296
Prob(Omnibus):                  0.196   Jarque-Bera (JB):                2.849
Skew:                           0.354   Prob(JB):                        0.241
Kurtosis:                       2.441   Cond. No.                     5.90e+07
==============================================================================

After learning a bunch from our first look at the data, I’m gearing up to do some fancier analyses in the coming weeks. We want to figure out even more about how different things are connected by using some advanced statistical methods like regression modeling and hypothesis testing.

This way, I can discover more detailed insights and get a better handle on how Boston’s economy really works. It’s like moving from the basics to the next level of understanding, using fancier tools to uncover more details in how key factors are linked.

Forecasting Future Trends Over Time Challenges in Predicting Future Trends Over Time:

Forecasting future trends over time comes with various challenges that can impact the accuracy and trustworthiness of predictions. One common hurdle is non-stationarity, where the statistical characteristics of the data change as time progresses. Dealing with dynamic environments, pinpointing outliers, and managing anomalies are critical challenges. Moreover, selecting suitable models that effectively capture intricate temporal patterns and accommodating irregularities in data distribution remains an ongoing issue. Successfully addressing these challenges emphasizes the need for robust techniques and meticulous preprocessing in applications involving the forecasting of future trends over time.

Applications of Predicting Future Trends Over Time:

The practice of predicting future trends over time has broad applications across diverse fields. In finance, it plays a crucial role in anticipating stock prices and currency exchange rates. Demand forecasting utilizes models that analyze time series data to estimate future product demand, facilitating efficient inventory management. Within the energy sector, forecasting is essential for predicting electricity consumption and optimizing the allocation of resources. Weather forecasting heavily relies on time series analysis to predict variables such as temperature and precipitation. These applications underscore the versatility of predicting future trends over time, providing valuable insights for decision-making across industries, from finance to logistics and beyond.

ARIMA

which stands for Auto Regressive Integrated Moving Average, is like a smart tool for predicting future values in a timeline, such as stock prices or weather patterns.

Auto Regressive (AR) Component: Think of this as looking at how today’s situation is connected to what happened in the past. If we say it’s AR(3), it means we’re looking at the connection between today and the past three days.

Integrated (I) Component: This part is about making sure our data is easy to work with. We do this by checking the difference between consecutive days. If we do it twice, it’s like looking at the change in change.

Moving Average (MA) Component: Here, we consider how today’s situation relates to any mistakes we made in predicting the past. If it’s MA(2), it means we’re looking at the connection between today and the errors we made in predicting the two previous days. The ARIMA model is just a combination of these three, written as ARIMA(p, d, q), where: p is about looking back in time. d is about making the data easy to work with. q is about learning from our past mistakes.

Steps in Building an ARIMA Model:

Inspecting Data: Look at how things have been changing over time. Making Things Simple: If things are too complicated, we simplify them by looking at differences between days.

Choosing Settings: Figure out how much we need to look back ( p), how many times to simplify ( d), and how much to learn from past mistakes ( q). Putting it All Together: Use these settings to build a smart model that learns from the past. Checking How Well It Works: See how well our model predicts the future by comparing it to new data.

Making Predictions: Once it’s working well, we can use our smart model to make predictions about what might happen next. ARIMA models are great for making predictions when things follow a clear pattern. However, for more complicated situations, there are other tools that might work even better.

Time series forecasting is like predicting the future based on how things have changed in the past. Imagine you have a timeline of events, and you want to figure out what might happen next. This is used in many areas, like predicting stock prices, estimating how much of a product people will want to buy, forecasting energy usage, or even predicting the weather.

Stationarity: We often assume that the way things have been changing will keep happening the same way. This makes it easier to make predictions.

Components of Time Series: Imagine the data as having three parts – a long-term trend (like a steady increase or decrease), repeating patterns (like seasons changing), and random ups and downs that we can’t explain (we call this noise). C

Common Models: There are different ways to make predictions. Some look at the past data’s trends and patterns, like ARIMA. Others, like LSTM and GRU, are like smart computer programs that learn from the past and make predictions.

Evaluation Metrics: We use tools to check if our predictions are accurate. It’s like making sure our guesses about the future match up with what actually happens.

Challenges: Sometimes things change in unexpected ways, or there are unusual events that we didn’t predict. Adapting to these changes is one of the tricky parts.

Applications: This forecasting tool is handy in many fields. For example, it can help predict how much money a company might make, how many products they need to produce, or how much energy a city might use. In a nutshell, time series forecasting helps us plan for the future by learning from the past. It’s like looking at patterns in a timeline to make smart guesses about what comes next.

Today’s analysis involves looking at patterns—like trends or recurring behaviors—and figuring out how they unfold over different time periods. Imagine you’re tracking data points at different moments.

The first step is to organize and visually represent this data, often with timestamps showing when each piece was recorded. This helps us see the patterns more clearly. Next, we use various methods to break down the data, like splitting it into different parts or figuring out how one data point relates to another. We might also choose specific models (like mathematical formulas) to better understand what’s happening in the data.

To make sure our understanding is accurate, we use measures like Mean Squared Error or Mean Absolute Error to check how well our models predict or represent the changing patterns. Once we’ve built and confirmed our model, we can use it to make predictions about future values or to understand what might happen next in those patterns. It’s crucial to keep an eye on things over time, updating our analysis as we get more data. This way, we can be sure our understanding stays relevant and reflects any new developments. The specific tools we use for this depend on what we’re studying and what we want to find out. There are computer programs, like pandas and states models in Python, that make these analyses easier for us to do.

Srijana Mada _MTH522-Project-2

Srijana Mada Mth522_Project-1

In today’s class,   providing us with insights into the practical significance of the observed difference in average ages between two distinct racial groups, namely the White and Black populations. Upon examination, I determined that the resulting Cohen’s d value was approximately 0.57. This numerical value holds particular significance as it enables us to gauge the effect size of the observed age difference. According to established guidelines, a Cohen’s d value of this magnitude falls into the category of a medium effect size.

What this essentially signifies is that the approximately 7-year difference in average ages between the White and Black racial groups carries meaningful weight. While it may not reach the magnitude of a large effect, it is nonetheless a noteworthy and discernible difference that merits our attention and consideration. In practical terms, this medium effect size implies that the disparity in average ages between these two racial groups is of moderate importance and relevance. It suggests that the age difference, while not overwhelmingly substantial, is statistically and practically significant, and should be taken into account when making relevant decisions or drawing conclusions in contexts where age plays a role.

Statistical Analysis:   today’s analysis, I applied two distinct statistical approaches—a two-sample t-test and a comprehensive Monte Carlo simulation—to assess potential age disparities between two groups, one represented by “AgesWhite” and the other by “AgesBlack.” Two-Sample T-Test: The two-sample t-test, a widely recognized statistical method, was utilized to ascertain whether there exists a statistically significant difference in means between the two groups. The results of the t-test are as follows: T-statistic: 19.21 P-value: 2.28e-79 Negative Log (base 2) of p-value: 261.24 The t-statistic value of 19.21 signifies a substantial difference in means between the ‘Black’ and ‘White’ racial groups. The remarkably small p-value (2.28e-79) provides strong evidence against the null hypothesis of no difference. The negative log (base 2) of the p-value accentuates the significance, illustrating that the observed age difference is equivalent to the likelihood of obtaining more than 261 consecutive tails when flipping a fair coin.

Remarkably, none of the 2,000,000 random samples generated in the Monte Carlo simulation produced a difference in means greater than the observed 7.2-year disparity between White and Black individuals. This outcome aligns with the t-test results, providing strong evidence that such a substantial age difference is exceedingly unlikely to occur by random chance if the null hypothesis (no difference in means) were true. Combined Conclusion: Both the two-sample t-test and the Monte Carlo simulation converge on a consistent conclusion. The age difference of 7.2 years between White and Black individuals is highly statistically significant. The t-test presents strong evidence against the null hypothesis, and the Monte Carlo simulation reinforces this by demonstrating the extreme unlikelihood of observing such a significant age difference by random chance. This collective statistical analysis firmly underscores the presence of a genuine and substantial disparity in mean ages between these two demographic groups.

Two-Sample T-Test Results:
T-statistic: 7.637258554192298
P-value: 4.715270331488705e-07

Monte Carlo Simulation Results:
Observed Age Difference: 9.2
P-value (Monte Carlo): 5e-06

 

In today’s analysis, I delved into examining the age distribution from various angles, aiming to understand its deviation from a standard normal distribution, particularly in terms of extremeness and behavior.

The initial phase of the analysis involved calculating the mean age and standard deviation, providing valuable insights into the distribution’s central tendency and variability. These statistical measures set the stage for further exploration. A key aspect of the analysis was the determination of what proportion of the right tail of the age distribution extended beyond 2 standard deviations from the mean, for both the Black and White races.

To accomplish this, a threshold was established by adding 2 times the standard deviation to the mean. This threshold served as a demarcation line, identifying outliers in the right tail of the distribution. Subsequently, I quantified the percentage of data points within the dataset that exceeded this threshold. This calculation shed light on the rarity of values in the right tail and provided a deeper understanding of the distribution’s characteristics. Moreover, as a point of reference, I utilized the standard normal distribution, allowing for a comparative analysis between our data and the theoretical normal distribution.

This comparison, particularly in the tail region extending beyond 2 standard deviations, facilitated an assessment of how the age distribution deviated from the idealized normal distribution.

As we delved into our dataset, we kept an eye out for things that seemed a bit unusual or didn’t quite fit the usual patterns. This led us to tweak our dataset to make sure it was ready for a closer look.

At the same time, I started making some educated guesses—basically, forming ideas that would help me figure out what to look at next. One interesting thought we had was whether how full hotels are might have something to do with how many people are working. It’s like a guess that guides us to focus on specific parts of the economic indicators I was checking out.

In the course of today’s analysis, an examination of the age distribution between the White and Black racial groups. I’ve a combined kernel density plot below for the ‘age’ variable in both the ‘Black’ and ‘White’ races. The kernel density plot representing the ‘age’ variable for the ‘Black’ race, shown in red, exhibits a positive skewness and a moderate level of peakness. According to the statistical summary, the dataset comprises 1,725 individuals with an average age of approximately 32.93 years and a standard deviation of roughly 11.39.

The age values in this dataset span from 13 to 88 years, and the quartile values offer insights into the distribution’s overall spread. On the other hand, the kernel density plot for the ‘age’ variable within the ‘White’ race, displayed in blue, displays a slight negative skewness and a relatively flat distribution.

The statistical summary for this dataset reveals a total of 3,244 individuals with an average age of approximately 40.13 years and a standard deviation of around 13.16. The age values in this dataset range from 6 to 91 years, and the quartile values provide valuable information about the distribution’s variability.