thesis for neural network

MIT Libraries home DSpace@MIT

• DSpace@MIT Home
• MIT Libraries
• Doctoral Theses

Modeling Intelligence via Graph Neural Networks

Terms of use

Date issued, collections.

• < Previous

Home > GRAD > THESIS > 402

Artificial Neural Network Concepts and Examples

Harcharan Kabbay , University of Missouri-St. Louis Follow

Document Type

Master of Arts

Mathematics

Date of Defense

Graduate advisor.

Adrian Clingher

Dr. Adrian Clingher, Ph.D

Dr. Qingtang Jiang, Ph.D

Dr. Haiyan Cai, Ph.D

Artificial Neural Networks have gained much media attention in the last few years. Every day, numer- ous articles on Artificial Intelligence, Machine Learning, and Deep Learning exist. Both academics and business are becoming increasingly interested in deep learning. Deep learning has innumerable uses, in- cluding autonomous driving, computer vision, robotics, security and surveillance, and natural language processing. The recent development and focus have primarily been made possible by the convergence of related research efforts and the introduction of APIs like Keras. The availability of high-speed compute resources such as GPUs and TPUs has also been instrumental in developing deep learning models.

While the development of the APIs like Keras offers a layer of abstraction and makes the model development convenient, the Mathematical logic behind the working of the Neural Networks is often misunderstood. The thesis focuses on the building blocks of a Neural Network in terms of Mathemat- ical terms and formulas. The research article also includes the details on the core parts of the Deep Learning algorithms like Forwardpropagation, Gradient Descent, and Backpropagation.

The research briefly covers the basic operations in Convolution Neural Networks, and a working example of multi-class classification problem using Keras library in R. CNN is a vast area of research in itself, and covering all the aspects of the ConvNets is out of scope of this paper. However, it provides an excellent foundation for understanding how Neural Networks work and how a CNN uses the concepts of the building blocks of a primary Neural Network in an image classification problem.

Recommended Citation

Kabbay, Harcharan, "Artificial Neural Network Concepts and Examples" (2022). Theses . 402. https://irl.umsl.edu/thesis/402

Since September 29, 2022

Included in

Other Mathematics Commons

Advanced Search

• Notify me via email or RSS
• Collections
• Disciplines

Participate

• Submission Guidelines
• Submit Research
• University of Missouri, St. Louis
• UMSL Library

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

File(s) under embargo

Reason: Working on submitting last chapter to journal/conference

until file(s) become available

On the use of $\alpha$-stable random variables in Bayesian bridge regression, neural networks and kernel processes.pdf

The first chapter considers the l_α regularized linear regression, also termed Bridge regression. For α ∈ (0, 1), Bridge regression enjoys several statistical properties of interest such

as sparsity and near-unbiasedness of the estimates (Fan & Li, 2001). However, the main difficulty lies in the non-convex nature of the penalty for these values of α, which makes an

optimization procedure challenging and usually it is only possible to find a local optimum. To address this issue, Polson et al. (2013) took a sampling based fully Bayesian approach to this problem, using the correspondence between the Bridge penalty and a power exponential prior on the regression coefficients. However, their sampling procedure relies on Markov chain Monte Carlo (MCMC) techniques, which are inherently sequential and not scalable to large problem dimensions. Cross validation approaches are similarly computation-intensive. To this end, our contribution is a novel non-iterative method to fit a Bridge regression model. The main contribution lies in an explicit formula for Stein’s unbiased risk estimate for the out of sample prediction risk of Bridge regression, which can then be optimized to select the desired tuning parameters, allowing us to completely bypass MCMC as well as computation-intensive cross validation approaches. Our procedure yields results in a fraction of computational times compared to iterative schemes, without any appreciable loss in statistical performance.

Next, we build upon the classical and influential works of Neal (1996), who proved that the infinite width scaling limit of a Bayesian neural network with one hidden layer is a Gaussian process, when the network weights have bounded prior variance. Neal’s result has been extended to networks with multiple hidden layers and to convolutional neural networks, also with Gaussian process scaling limits. The tractable properties of Gaussian processes then allow straightforward posterior inference and uncertainty quantification, considerably simplifying the study of the limit process compared to a network of finite width. Neural network weights with unbounded variance, however, pose unique challenges. In this case, the classical central limit theorem breaks down and it is well known that the scaling limit is an α-stable process under suitable conditions. However, current literature is primarily limited to forward simulations under these processes and the problem of posterior inference under such a scaling limit remains largely unaddressed, unlike in the Gaussian process case. To this end, our contribution is an interpretable and computationally efficient procedure for posterior inference, using a conditionally Gaussian representation, that then allows full use of the Gaussian process machinery for tractable posterior inference and uncertainty quantification in the non-Gaussian regime.

Finally, we extend on the previous chapter, by considering a natural extension to deep neural networks through kernel processes. Kernel processes (Aitchison et al., 2021) generalize to deeper networks the notion proved by Neal (1996) by describing the non-linear transformation in each layer as a covariance matrix (kernel) of a Gaussian process. In this way, each succesive layer transforms the covariance matrix in the previous layer by a covariance function. However, the covariance obtained by this process loses any possibility of representation learning since the covariance matrix is deterministic. To address this, Aitchison et al. (2021) proposed deep kernel processes using Wishart and inverse Wishart matrices for each layer in deep neural networks. Nevertheless, the approach they propose requires using a process that does not emerge from the limit of a classic neural network structure. We introduce α-stable kernel processes (α-KP) for learning posterior stochastic covariances in each layer. Our results show that our method is much better than the approach proposed by Aitchison et al. (2021) in both simulated data and the benchmark Boston dataset.

Degree Type

• Doctor of Philosophy

Campus location

• West Lafayette

Advisor/Supervisor/Committee Chair

Additional committee member 2, additional committee member 3, additional committee member 4, usage metrics.

• Computational statistics
• Spatial statistics
• Statistical theory

Estimating the State of Health of Lithium Ion Batteries using Neural Networks

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Help | Advanced Search

Computer Science > Machine Learning

Title: bridging the fairness divide: achieving group and individual fairness in graph neural networks.

Abstract: Graph neural networks (GNNs) have emerged as a powerful tool for analyzing and learning from complex data structured as graphs, demonstrating remarkable effectiveness in various applications, such as social network analysis, recommendation systems, and drug discovery. However, despite their impressive performance, the fairness problem has increasingly gained attention as a crucial aspect to consider. Existing research in graph learning focuses on either group fairness or individual fairness. However, since each concept provides unique insights into fairness from distinct perspectives, integrating them into a fair graph neural network system is crucial. To the best of our knowledge, no study has yet to comprehensively tackle both individual and group fairness simultaneously. In this paper, we propose a new concept of individual fairness within groups and a novel framework named Fairness for Group and Individual (FairGI), which considers both group fairness and individual fairness within groups in the context of graph learning. FairGI employs the similarity matrix of individuals to achieve individual fairness within groups, while leveraging adversarial learning to address group fairness in terms of both Equal Opportunity and Statistical Parity. The experimental results demonstrate that our approach not only outperforms other state-of-the-art models in terms of group fairness and individual fairness within groups, but also exhibits excellent performance in population-level individual fairness, while maintaining comparable prediction accuracy.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Xpeng Motors

• Beijing Auto show

XPeng at Beijing Auto Show: 2K pure vision ADAS, neural network, 1km/sec fast charging, and a new AI-driven EV sub-brand [Video]

XPeng Motors kept things relatively short but sweet during its 20-minute presentation at the 2024 Beijing International Auto Show earlier today, but there’s a lot to get excited about following several updates from XPeng founder, chairman, and CEO He Xiaopeng, including an “entirely new breed” of EVs under a new sub-brand. Here’s the full recap.

Today’s presentation in front of a crowd in Beijing (you can view it in its entirety below) started off simply recapping much of the same news we reported on in 2024, some of it dating back to the Chinese automaker’s annual Tech Day in October 2023.

XPeng CEO He Xiaopeng spoke in front of an X9 multi-purpose vehicle (MPV), a growing segment of luxury minivans in China. Xiaopeng highlighted much of the early success of XPeng’s first MPV model, which is versatile in that it can be configured to seat seven passengers or four, with room to transport five bikes.

As a popular BEV model amongst Chinese celebrities and athletes, XPeng’s CEO used the X9 as the vessel to highlight some of the advanced technologies it has been working on, including expansions of its XNGP ADAS technology, including new AI Valet and bodyguard functions. The automaker’s founder and chairman spoke to these technologies and what they mean for the future of EVs:

We are proud to demonstrate XPeng’s technological innovation prowess, through which we are laying a pathway to greater inclusion and equality in smart mobility. The next decade will be a ‘golden decade’ of smart vehicles. The core of smart vehicle advances is how to operate with automative software adoption emerging as the new industry norm. Looking ahead, XPeng will roll out the on-road testing of AI-powered functions integrated into XPeng models.

To support XNGP and other ADAS functions , XPeng used the Beijing Auto Show to share plans to deploy what it calls the “industry’s first mass-produced 2K pure visual neural network large model in vehicles.” This news confirms previous rumors we reported that XPeng was abandoning LiDAR sensors in favor of pure vision, similar to Tesla FSD.

These upgrades to perception and planning/control models will utilize over two million high-definition grids to reconstruct worlds around XPeng BEVs, ensuring that any and all surrounding objects and obstacles are identified quickly and effectively. The new technology is further supported by neural-network-based planning models, which can learn, think, and perform actions like the human mind.

According to the Beijing press conference , such neural technologies enable XPeng to deliver more human-like, self-learning vehicles that will rely heavily on AI moving forward. That includes the automaker’s latest operating system, XOS 5.1.0, which delivers several new AI-powered features to debut in the X9 before reaching other eligible XPeng EVs on May 20, 2024.

Those updates include the previously mentioned AI Valet Driver, upgraded surround reality (SR) perception capabilities, ask expanded function and learning capabilities of the automaker’s in-car AI assistant. We recommend checking out the video below for a real-world view of this technology being demonstrated.

XPeng’s new sub-brand will be called MONA

Last month, we shared news that XPeng had plans for a new EV sub-brand that focused heavily on artificial intelligence, as mentioned above, and well beyond. During the recent China Electric Vehicle 100 Summit, XPeng Chairman and CEO He Xiaopeng vowed to invest RMB 3.5 billion (~$492M) in the automaker’s “AI-enabled smart driving” technology in 2024 for R&D and the hiring of 4,000 new employees.

Xiaopeng also said the next decade of EVs will be one of intelligence and smart driving technology. As such, the new brand was in the works to deliver AI-centric tech at an affordable price for all, targeting MSRPs around RMB 100,000-150,000 ($14,000-$21,000).

At the time, we reported the unnamed sub-brand would launch in China soon as XPeng promised it will “create a new breed of AI-powered Smart EVs for young customers worldwide.” Today in Beijing, XPeng confirmed the new sub-brand is called MONA, which stands for “Made Of New AI.”

The company’s CEO said MONA will officially be introduced this June, so stay tuned for more details on that.

Other XPeng updates from Beijing and the video footage

Last but not least, XPeng shared updates in regard to its charging technology, low-altitude flying car arm AeroHT, and its recent cooperation agreement signed with Volkswagen Group . On the charging side of things, XPeng says it is planning to upgrade its 800 kW DC fast chargers in Q3 2024, enabling what could potentially be the best charge speeds in the industry.

The automaker says the upgrades to the facilities will enable XPeng EVs to add more than 1 km (.62 miles) per second. AeroHT’s flying car was on display next to other XPeng EVs in Beijing after turning plenty of heads at CES in January. The eVTOL arm’s other vehicle, the modular flying car , is still seeking airworthiness certification and is expected to begin pre-sales in China in Q4 2024.

That’s all for now. As promised, here is the full XPeng press conference from the Beijing Auto Show, translated to English:

FTC: We use income earning auto affiliate links. More.

Scooter Doll is a writer, designer and tech enthusiast born in Chicago and based on the West Coast. When he’s not offering the latest tech how tos or insights, he’s probably watching Chicago sports. Please send any tips or suggestions, or dog photos to him at [email protected]