机器学习中的贝叶斯网络

2025年6月17日 | 阅读时长8分钟

贝叶斯网络是一种概率图模型，它使用有向无环图 (DAG) 表示变量集之间的条件依赖关系。图的每个节点代表一个随机变量，而边则表示这些随机变量之间的条件依赖关系。贝叶斯网络在机器学习领域中被广泛使用，因为它能够在不确定性下进行推理，并提供联合概率分布的高效表示和计算。

在建模复杂变量关系时，有时即使数据部分缺失，也可以针对本项目描述贝叶斯网络，它允许以依赖关系的形式表示紧凑性，将图论和概率论结合起来。在医疗诊断系统这样的情况下，贝叶斯网络可以建模疾病、测试结果和症状之间存在的关系，从而可以根据观察到的症状预测某种疾病的可能性。

贝叶斯网络在机器学习中的应用具有优势，因为它们可以包含先验知识并在新数据到来时更新信念。这个贝叶斯定理允许它根据证据修正其概率。因此，贝叶斯网络非常适合用于决策系统、异常检测、自然语言处理和推荐系统等应用。

构建贝叶斯网络需要定义 DAG 的结构，为每个节点指定条件概率表，然后执行推理。贝叶斯网络中的推理用于在给定其他变量证据的情况下计算某些变量的后验概率。执行此操作的算法包括变量消除、信念传播或马尔可夫链蒙特卡洛 (MCMC) 等采样方法。

现在我们将使用贝叶斯网络来检测异常。

导入库

 
# data manipulation 
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split  # Seperate train-test samples
from sklearn import preprocessing  # Scaling data
from sklearn.decomposition import PCA  # PCA implementation
#plotly fix
from plotly.offline import init_notebook_mode
init_notebook_mode(connected=True)
# visualization 
import matplotlib.pyplot as plt   # Plots
import plotly.express as px # PCA plots 
import seaborn as sns # Plots
import matplotlib as mpl
from cycler import cycler   # can help with assigning colors
from tabulate import tabulate # Creates tables
from sklearn.ensemble import ExtraTreesClassifier # Decision trees - used for feature importance
import matplotlib.lines as mlines # For use with legends, triangles, stars, and other markers

from tensorflow.keras.layers import Dense, Lambda
from tensorflow.keras import Sequential, Input
from tensorflow.keras.backend import clear_session
from tensorflow.keras.optimizers import Adam
from keras import metrics
from sklearn.metrics import confusion_matrix
from sklearn.preprocessing import OneHotEncoder
from keras import regularizers
import tensorflow as tf
import tensorflow_probability as tfp   

加载数据集

 
# Importing the  Dataset
wine = pd.read_csv("./wine.csv")

# Descriptive Statistics
pd.set_option('display.expand_frame_repr', False)

one_sum = pd.DataFrame(round(wine.groupby("type").describe().stack(),3))
one_sum   

输出

EDA

 
plt.figure(figsize = (20,15))
plt.title("Histograms per variable and class on the wine dataset")
sns.set_theme(style="dark")
colms = ["fixed.acidity","volatile.acidity","citric.acid",
            "residual.sugar",
            "chlorides",
            "free.sulfur.dioxide",
            "total.sulfur.dioxide",
            "density",
            "pH",
            "sulphates","alcohol","quality","type"]
mments = ["(g(tartaric acid)/dm^3)", "(g(acetic acid)/dm^3)", 
               "(g/dm^3)", "(g/dm^3)", "(g(sodium chloride)/dm^3)",
                "(mg/dm^3)", "(g/cm^3)","","(g(potassium sulphate)/dm^3)",
                "(%)","","",""
               ]
i = 1
j = 0
for colms in colms:
    plt.subplot(4,4,i)
    sns.histplot(data = wine, x = colms, hue = "type", palette = "tab10")
    plt.xlabel("")
    plt.title("{} {}".format(colms,mments[j]))
    j = j+1
    i = i+1   

输出

 
"""
Uses Seaborn to create a heatmap of the correlation matrix.
"""
plt.figure(figsize = (12,10))
ax = plt.axes()

sns.heatmap(wine.drop(["type"], axis = 1).corr(), annot =True, cmap="PiYG", ax = ax)
ax.set_title("Heatmap of Correlation Matrix - Wine Dataset", loc = "left", fontsize = 15,
            fontweight ="bold")   

输出

 
# Preprocessing the data


# Classification task

# Features do not include quality, and the goal is type.
features= ["fixed.acidity",
                 "volatile.acidity",
                 "citric.acid",
                 "residual.sugar",
                 "chlorides",
                 "free.sulfur.dioxide",
                 "total.sulfur.dioxide",
                 "density","pH",
                 "sulphates","alcohol"]
class_wine = wine.loc[:,features]
class_type = wine.loc[:,['type']]



# Regression task

reg_wine = wine.loc[:,features]
quality = wine.loc[:,["quality"]]   

让我们看看特征的重要性。

 
"""
Below is an implementation of the scikit-learn additional trees classifier for both regression and classification applications.

random_state = 4 is used to make the results reproducible
"""


# Classification Imporance

Etc = ExtraTreesClassifier(random_state = 4)
Etc.fit(class_wine,class_type.values.ravel())
print(Etc.feature_importances_) #employ the built-in class feature_importances of classifiers that are based on trees.

#Plotting feature importance graphs for improved visualization
feat_importances = pd.Series(Etc.feature_importances_, index=class_wine.colms)
feat_importances.nlargest(10).plot(kind='barh')
plt.title("Feature Importance for predicting type of wine (Extra-Trees Method)", loc = "left", fontsize = 10,
            fontweight ="bold")
plt.xlabel("Score")
plt.show()

# Regression Importance

Etc = ExtraTreesClassifier(random_state = 4)
Etc.fit(reg_wine,quality.values.ravel())
print(Etc.feature_importances_) # employ the built-in class feature_importances of classifiers that are based on trees.

# Plotting feature importance graphs for improved visualization
feat_importances = pd.Series(Etc.feature_importances_, index=reg_wine.colms)
feat_importances.nlargest(10).plot(kind='barh')
plt.title("Feature Importance for predicting quality of wine (Extra-Trees Method)",loc = "left", fontsize = 10,
            fontweight ="bold") 
plt.xlabel("Score")
plt.show()   

输出

在重叠区域之间存在模糊区域的情况下，三种葡萄酒可以清楚地区分开来。

 
"Section in which pca uses and scales classification data"

# Classification - Scale inputs

class_wine = class_wine.values
class_type = class_type.values

class_wine = preprocessing.scale(class_wine, axis=0, with_mean=True, with_std=True, copy= False)

"""
Below, we use the sklearn:: PCA function to define a 2-PCA job.

A new dataframe (wine_one_pca) is then created with the corresponding 2 PC components. 

DISCLAIMER: The R-ggplot2 equivalent of this graph was ultimately utilized in my report, even though variants of it are implemented below. 
Although the code for its creation is included in the appendix section, it is also included below.


______________________________
{R CODE FOR PCA PLOT IN REPORT}

library(ggplot2)  # package for visualization
library(ggfortify) # Look at PCA with loadings.


wine = read.csv("./wine.csv")

wine.pca <- prcomp(wine[,c(1:(ncol(wine)-2))], center = TRUE,scale. = TRUE)
autoplot(wine.pca, data = wine, colour = "type", loadings = TRUE,
        loadings.label = TRUE, loadings.label.size = 3)
______________________________
"""

# Defining PCA
pca = PCA(n_components = 2)
pca_wine = pca.fit_transform(class_wine)


# Make a dataframe for visualization.
wine_one_pca = pd.DataFrame(data = pca_wine, 
                     colms = ["PC 1", "PC 2"])
wine_one_pca = pd.concat([wine_one_pca, wine[["type"]]], axis = 1)

# To be used on the title, explained variance
pca2_variance = pca.explained_variance_ratio_.sum()*100

# Plotting parameters
fig, ax = plt.subplots(1,1,figsize= (8,8))
ax.set_xlabel('Principal Component 1', fontsize = 15)
ax.set_ylabel('Principal Component 2', fontsize = 15)
ax.set_title(f'2 component PCA: Explained Variance: {pca2_variance:.2f}%', fontsize = 20)

# Set 1 color per type
targets = [1,2,3]
colors = ["b","g","r"]
for target, color in zip(targets,colors):
    indicesToKeep = wine_one_pca['type'] == target
    ax.scatter(wine_one_pca.loc[indicesToKeep, 'PC 1']
               , wine_one_pca.loc[indicesToKeep, 'PC 2']
               , c = color
               , s = 50
              )
ax.legend(targets)
ax.grid()   

输出

为了避免过拟合，我们将数据集进行分割。

 
# Split test and train data
"""
Permit the dataset to be divided into train and test counterparts.
Reproducible results are achieved by using random state = 42.

X_xxxx_class: Features per xxxx (train or test) dataset.
Y_xxxx_class: Target per xxxx dataste in one Hot encoded mode (sparse = False to return an array)
"""

# Main train-test datasets
train_class, test_class = train_test_split(wine, test_size=0.2, random_state = 42)

# Data preprocessing
train_X_class = train_class.drop(["type","quality"], axis = 1)   # Ntrainx11
train_Y_class = OneHotEncoder(sparse=False).fit_transform(train_class[["type"]].values) # Ntrainx3

test_X_class = test_class.drop(["type","quality"], axis = 1)  # Ntestx11
test_Y_class = OneHotEncoder(sparse=False).fit_transform(test_class[["type"]].values) # Ntestx3

d = train_X_class.shape[1]  # second index = 11
batch_size = len(train_X_class)  # = N train. The whole training dataset used per epoch

# Print shapes / dataset lengths
print(train_X_class.shape, train_Y_class.shape)
print(test_X_class.shape, test_Y_class.shape)

# Input scaling to enhance loss convergence
train_X_class = preprocessing.scale(train_X_class, axis=0, with_mean=True, with_std=True, copy= False)
test_X_class = preprocessing.scale(test_X_class, axis=0,with_mean=True,with_std = True, copy = False)   

输出

该报告将检查基线模型并找出其缺点。

 
# Set Seed value for reproducibility

seed_value = 0
"""
The frequentist NN was implemented using Keras. Additionally, pertinent metrics and accuracies are presented.


The following section defines a function to visualize the confusion matrix.
"""

# making NN reproducible
np.random.seed(seed_value)
tf.random.set_seed(seed_value)

# Creating the  model 
model_cl = Sequential()
model_cl.add(Input(shape = (11)))
model_cl.add(Dense(5, kernel_regularizer = regularizers.l2(0.001), activation = "relu"))  # hidden layers. a L2 regularised was used
model_cl.add(Dense(3, activation = "softmax"))

# assemble and adjust with 10% * train_test = test of validation
model_cl.compile(optimizer=Adam(learning_rate=0.01), 
                    loss='categorical_crossentropy',
                   metrics = ["acc",metrics.RootMeanSquaredError(),metrics.CategoricalCrossentropy()]
                   )

history = model_cl.fit(x=train_X_class, y=train_Y_class, validation_split = 0.1,
                         epochs=200, batch_size=batch_size, verbose = 0)

# Estimate the Y fit in the train and measure results.
train_pred= model_cl.predict(train_X_class)
scores = model_cl.evaluate(train_X_class, train_Y_class, verbose=0)
print('Accuracy on training data: {}% \n Error on training data: {}'.format(round(scores[1],3), round(scores[0],3)))   

# Estimate the Y fit of the test and measure the results.
test_pred= model_cl.predict(test_X_class)
two_scores = model_cl.evaluate(test_X_class, test_Y_class, verbose=0)
print('Accuracy on test data: {}% \n Error on test data: {}'.format(round(two_scores[1],3), round(two_scores[0],3)))    

# Testing the data confusion matrix
confusion_matrix(np.argmax(test_Y_class, axis=1),       # Input class membership
                 np.argmax(test_pred, axis=1))  # Output class membership
   

输出

 
# Function for heatmaps
def heatmap(pred, text, scores):
    
    """
    Develops a visualization feature for test data epoch_finals.


    Input:
    pred: prediction vector based on inputs from the test dataset. The source is model.predict(test_X)
    text: depending on the mode, this text will be used in the story title.
    results: output accuracy results (from model.evaluate(test_X,test_Y))


    """

    plt.figure(figsize = (5,5))
    ax = plt.axes()

    sns.heatmap(confusion_matrix(np.argmax(test_Y_class, axis=1),       # Input class membership
                 np.argmax(pred, axis=1)),annot =True, cmap="PiYG", fmt='g')
    ax.set_title("Confusion Matrix - {}".format(text))
    ax.set_xlabel("Predicted Value")
    ax.set_ylabel("True Value")
    plt.text(0.45,0.01,
         "Acurracy on test data:{}%".format(round(100*scores[1],2)),
         horizontalalignment='center',
         verticalalignment='center',
         transform=plt.gcf().transFigure,
         fontsize = 13
            )
"Output of heatmap function presented above"

heatmap(pred_test, "NN", scores2)   

输出

 
"""
Another function is added to visualize the loss and accuracy metric throughout the epochs. It plots the loss function using the history object as an input.
Additionally, it plots the accuracy over time if the "acc" is true. 
"""

def loss_acc_plotting(x, acc = True):
   
    """
    loss_acc_plotting:
    A straightforward function that charts accuracy and loss measures over time
    
    Input: 
  x: The object (often the history variable) to depict the loss function
    If accuracy should also be plotted, set acc: to True.
    
    Output:
    Respective plots
    """
    history = x.history

    values_loss = history["loss"]
    val_values_loss = history["val_loss"]

    epoch_final = len(history["loss"])+1
    epochs = range(1,epoch_final)

    plt.plot(epochs, values_loss, "b", label = "Training Loss",linewidth=2)
    plt.plot(epochs, val_values_loss,"r", label = "Validation Loss", linewidth=2)
    plt.title("Training and Validation Loss")
    plt.xlabel("Epochs")
    plt.ylabel("Loss")
    plt.legend()

    plt.show()
    
    if acc == True:
        plt.clf()
    
        acc_values = history['acc']
        val_acc_values = history['val_acc']
        plt.plot(epochs, acc_values, 'b', label='Training acc',linewidth=2)
        plt.plot(epochs, val_acc_values, 'r', label='Validation acc',linewidth=2)
        plt.title('Training and validation accuracy')
        plt.xlabel('Epochs')
        plt.ylabel('Accuracy (%)')
        plt.legend(loc = "right")
        plt.show()
# To construct plots on a frequentist NN, call the created function.
loss_acc_plotting(history, acc= True)   

输出

贝叶斯网络

权重向量将服从具有可训练参数的正态后验分布，以及均值为 0 且方差为 1 的正态先验分布。

 
"""
The previous model will now be converted to a Bayesian model using keras.


Normal priors and posterior are set appropriately, whereas DenseFlipout layers are employed.


A divergence mechanism between these two is also defined by the fn_divergence_kl function.


Loss functions and optimizers are employed as before.
"""

np.random.seed(seed_value)
tf.random.set_seed(seed_value)

# Use of the KL divergence function as an input for flipout layers
fn_divergence_kl=lambda q, p, _: tfp.distributions.kl_divergence(q, p) / (train_X_class.shape[0] *1.0)

baynet = Sequential()
baynet.add(Input(shape = 11))
baynet.add(tfp.layers.DenseFlipout( units =20,
                         kernel_prior_fn = tfp.layers.default_mean_field_normal_fn(),
                         kernel_posterior_fn = tfp.layers.default_mean_field_normal_fn() ,
                         kernel_divergence_fn = fn_divergence_kl,
                         activation = "relu"
                        )
       )

baynet.add(tfp.layers.DenseFlipout(units = 3, 
                                kernel_prior_fn = tfp.layers.default_mean_field_normal_fn(),
                                kernel_posterior_fn = tfp.layers.default_mean_field_normal_fn(), 
                                kernel_divergence_fn = fn_divergence_kl,
                                activation = "softmax")
       )


baynet.compile( optimizer = "adam", loss = 'categorical_crossentropy',
           metrics = ["acc", ]
           ) 

baynet_history = baynet.fit(train_X_class,train_Y_class, epochs = 200, validation_split = 0.1, 
                      batch_size = batch_size,
                      verbose = 0
                     )

# The original loss_acc_plotting function was originally published.
loss_acc_plotting(baynet_history, acc= True)

# Estimate the Y fit in the train and measure results.
train_pred_vi= baynet.predict(train_X_class)
scores_vi = baynet.evaluate(train_X_class, train_Y_class, verbose=0)
print('Accuracy on training data: {}% \n Error on training data: {}'.format(scores_vi[1],scores_vi[0]))   

# Estimate the Y fit in the test and measure the results.
test_vi_pred= baynet.predict(test_X_class)
vi_two_scores= baynet.evaluate(test_X_class, test_Y_class, verbose=0)
print('Accuracy on test data: {}% \n Error on test data: {}'.format(two_scores_vi[1],  two_scores_vi[0]))    

# Fitted values, or results anticipated from training data
confusion_matrix(np.argmax(test_Y_class, axis=1),       #  membership of input class
                 np.argmax(test_vi_pred, axis=1))  # membership of input class   

输出

下一主题机器学习中的成本函数

← 上一个下一个 →

机器学习中的贝叶斯网络

导入库

加载数据集

EDA

贝叶斯网络

联系信息

关注我们

教程

面试题

在线编译器

Python

Java

.Net Framework

AI, ML and Data Science

Cloud Technology

B.Tech and MCA

Web Technology

PHP

Software Testing

Technical Interview

Java Interview

Python

Web Interview

Database Interview

B.Tech / MCA

Important Interview

Software Testing Interview

Company Interviews

Online Compilers

Multiple Choice Questions

机器学习

监督式学习

分类

杂项

相关教程

面试题

机器学习中的贝叶斯网络

导入库

加载数据集

EDA

贝叶斯网络

相关帖子

LiDAR：用于 3D 重建的光探测与测距

递归特征消除

什么是知识图谱

处理不平衡数据的分类

Mini Batch K-means 聚类算法

人类行为识别

机器学习中的数据可视化工具

谱共聚类

CycleGAN

广义估计方程简介

订阅 Tpoint Tech

联系信息

关注我们

教程

面试题

在线编译器