一维数据中的锐步检测

Question

我的问题基本上分为两部分。我有一个一维数据集，我希望 一方面平滑 以消除噪音，另一方面我想非常精确地检测（大）步骤 strong> 在数据中。

1) 关于步数检测部分，我已经在这里找到了一个有趣的条目Step detection in one-dimensional data，但这种方法似乎不适用于我的数据。

所以我的原始数据如下所示： enter image description here

当我应用上述方法时，结果如下： enter image description here

2) 关于数据平滑部分，我已经尝试了一些数据平滑，例如应用 scipy 中的 gaussian_filter1d。但是当然，我的数据越平滑，检测到的步骤就越不精确（或者我在这部分错了吗？）。有没有更好的方法来平滑数据但保持步长清晰？

原始数据为：

[441.0, 414.0, 389.0, 430.0, 403.0, 402.0, 475.0, 417.0, 535.0, 476.0, 461.0, 415.0, 476.0, 442.0, 411.0, 422.0, 417.0, 451.0, 428.0, 455.0, 446.0, 414.0, 411.0, 413.0, 412.0, 424.0, 454.0, 439.0, 422.0, 417.0, 449.0, 417.0, 445.0, 411.0, 388.0, 420.0, 432.0, 457.0, 421.0, 415.0, 486.0, 449.0, 419.0, 453.0, 424.0, 451.0, 432.0, 464.0, 430.0, 469.0, 404.0, 461.0, 443.0, 440.0, 407.0, 452.0, 424.0, 426.0, 451.0, 423.0, 479.0, 407.0, 419.0, 435.0, 463.0, 455.0, 420.0, 413.0, 441.0, 451.0, 398.0, 433.0, 449.0, 451.0, 441.0, 450.0, 412.0, 440.0, 419.0, 408.0, 393.0, 395.0, 409.0, 450.0, 416.0, 398.0, 443.0, 403.0, 420.0, 411.0, 397.0, 416.0, 455.0, 392.0, 441.0, 422.0, 398.0, 445.0, 491.0, 405.0, 410.0, 381.0, 427.0, 403.0, 408.0, 427.0, 441.0, 464.0, 429.0, 420.0, 463.0, 405.0, 422.0, 408.0, 421.0, 464.0, 442.0, 429.0, 477.0, 407.0, 438.0, 453.0, 359.0, 435.0, 452.0, 443.0, 371.0, 503.0, 464.0, 466.0, 446.0, 465.0, 409.0, 440.0, 415.0, 384.0, 462.0, 446.0, 454.0, 440.0, 490.0, 400.0, 430.0, 416.0, 466.0, 436.0, 451.0, 408.0, 423.0, 449.0, 396.0, 417.0, 409.0, 446.0, 408.0, 426.0, 401.0, 417.0, 410.0, 361.0, 381.0, 416.0, 443.0, 413.0, 449.0, 411.0, 425.0, 439.0, 392.0, 464.0, 432.0, 387.0, 408.0, 454.0, 455.0, 449.0, 420.0, 379.0, 446.0, 400.0, 436.0, 461.0, 451.0, 433.0, 417.0, 415.0, 449.0, 397.0, 390.0, 381.0, 416.0, 411.0, 407.0, 420.0, 474.0, 444.0, 390.0, 413.0, 410.0, 428.0, 425.0, 405.0, 398.0, 406.0, 431.0, 420.0, 468.0, 463.0, 431.0, 415.0, 381.0, 427.0, 438.0, 431.0, 388.0, 440.0, 467.0, 453.0, 441.0, 373.0, 418.0, 450.0, 395.0, 407.0, 433.0, 425.0, 462.0, 392.0, 424.0, 435.0, 429.0, 409.0, 430.0, 410.0, 394.0, 407.0, 412.0, 420.0, 432.0, 419.0, 415.0, 338.0, 397.0, 436.0, 422.0, 409.0, 431.0, 422.0, 437.0, 377.0, 493.0, 417.0, 416.0, 468.0, 468.0, 428.0, 459.0, 410.0, 462.0, 391.0, 433.0, 410.0, 388.0, 397.0, 421.0, 416.0, 429.0, 410.0, 447.0, 368.0, 397.0, 405.0, 421.0, 393.0, 446.0, 404.0, 414.0, 418.0, 426.0, 408.0, 389.0, 426.0, 442.0, 476.0, 409.0, 426.0, 392.0, 402.0, 435.0, 415.0, 392.0, 435.0, 442.0, 461.0, 422.0, 389.0, 445.0, 423.0, 379.0, 435.0, 416.0, 454.0, 490.0, 446.0, 457.0, 432.0, 413.0, 407.0, 465.0, 451.0, 427.0, 459.0, 446.0, 419.0, 413.0, 418.0, 409.0, 413.0, 442.0, 454.0, 405.0, 397.0, 445.0, 441.0, 457.0, 390.0, 454.0, 403.0, 444.0, 422.0, 475.0, 448.0, 410.0, 462.0, 421.0, 418.0, 436.0, 406.0, 401.0, 414.0, 435.0, 371.0, 455.0, 438.0, 421.0, 438.0, 424.0, 422.0, 408.0, 430.0, 382.0, 366.0, 466.0, 424.0, 405.0, 394.0, 407.0, 429.0, 443.0, 458.0, 439.0, 453.0, 403.0, 430.0, 370.0, 393.0, 396.0, 447.0, 441.0, 426.0, 401.0, 381.0, 454.0, 415.0, 474.0, 455.0, 456.0, 406.0, 457.0, 436.0, 399.0, 394.0, 412.0, 383.0, 382.0, 423.0, 417.0, 426.0, 419.0, 414.0, 412.0, 349.0, 407.0, 397.0, 454.0, 415.0, 383.0, 398.0, 411.0, 378.0, 425.0, 423.0, 283.0, 288.0, 290.0, 275.0, 271.0, 293.0, 264.0, 261.0, 260.0, 251.0, 266.0, 383.0, 446.0, 410.0, 367.0, 487.0, 411.0, 392.0, 370.0, 414.0, 411.0, 424.0, 374.0, 427.0, 439.0, 425.0, 432.0, 425.0, 383.0, 388.0, 391.0, 367.0, 441.0, 476.0, 414.0, 491.0, 444.0, 467.0, 431.0, 459.0, 433.0, 439.0, 438.0, 431.0, 396.0, 419.0, 452.0, 468.0, 436.0, 476.0, 445.0, 455.0, 428.0, 430.0, 397.0, 443.0, 411.0, 446.0, 508.0, 428.0, 409.0, 280.0, 327.0, 327.0, 321.0, 282.0, 288.0, 304.0, 286.0, 292.0, 293.0, 306.0, 319.0, 287.0, 309.0, 298.0, 276.0, 271.0, 300.0, 299.0, 292.0, 318.0, 295.0, 315.0, 307.0, 277.0, 286.0, 289.0, 310.0, 289.0, 292.0, 312.0, 502.0, 455.0, 460.0, 422.0, 392.0, 395.0, 370.0, 380.0, 451.0, 409.0, 428.0, 429.0, 429.0, 482.0, 444.0, 456.0, 423.0, 446.0, 465.0, 384.0, 437.0, 487.0, 421.0, 396.0, 432.0, 400.0, 394.0, 397.0, 434.0, 475.0, 467.0, 393.0, 410.0, 425.0, 408.0, 424.0, 418.0, 473.0, 432.0, 459.0, 417.0, 475.0, 433.0, 460.0, 470.0, 423.0, 414.0, 404.0, 453.0, 478.0, 389.0, 448.0, 429.0, 438.0, 381.0, 432.0, 404.0, 445.0, 421.0, 460.0, 440.0, 427.0, 386.0, 443.0, 447.0, 403.0, 439.0, 437.0, 411.0, 376.0, 495.0, 430.0, 412.0, 459.0, 457.0, 318.0, 292.0, 308.0, 299.0, 302.0, 276.0, 274.0, 294.0, 302.0, 295.0, 270.0, 333.0, 285.0, 291.0, 292.0, 288.0, 304.0, 262.0, 289.0, 277.0, 299.0, 293.0, 313.0, 291.0, 337.0, 285.0, 288.0, 291.0, 297.0, 285.0, 305.0, 297.0, 283.0, 287.0, 281.0, 312.0, 356.0, 282.0, 301.0, 301.0, 300.0, 276.0, 292.0, 281.0, 299.0, 316.0, 275.0, 306.0, 302.0, 296.0, 280.0, 283.0, 294.0, 303.0, 290.0, 309.0, 301.0, 296.0, 287.0, 279.0, 281.0, 277.0, 318.0, 292.0, 286.0, 319.0, 289.0, 329.0, 290.0, 300.0, 297.0, 300.0, 296.0, 300.0, 288.0, 322.0, 292.0, 290.0, 277.0, 294.0, 279.0, 286.0, 291.0, 298.0, 273.0, 276.0, 302.0, 308.0, 292.0, 327.0, 297.0, 276.0, 319.0, 277.0, 283.0, 279.0, 289.0, 315.0, 277.0, 281.0, 283.0, 274.0, 290.0, 377.0, 519.0, 406.0, 508.0, 405.0, 426.0, 440.0, 411.0, 411.0, 428.0, 423.0, 417.0, 428.0, 425.0, 401.0, 451.0, 435.0, 414.0, 409.0, 446.0, 413.0, 433.0, 397.0, 447.0, 456.0, 412.0, 412.0, 423.0, 445.0, 449.0, 414.0, 420.0, 419.0, 422.0, 408.0, 436.0, 422.0, 421.0, 468.0, 435.0, 464.0, 406.0, 282.0, 294.0, 305.0, 278.0, 291.0, 272.0, 282.0, 284.0, 333.0, 304.0, 284.0, 290.0, 309.0, 293.0, 300.0, 295.0, 280.0, 295.0, 268.0, 293.0, 300.0, 296.0, 279.0, 282.0, 293.0, 290.0, 302.0, 308.0, 300.0, 285.0, 283.0, 273.0, 274.0, 281.0, 316.0, 294.0, 271.0, 301.0, 297.0, 278.0, 305.0, 310.0, 293.0, 284.0, 307.0, 279.0, 309.0, 287.0, 301.0, 289.0, 275.0, 309.0, 289.0, 290.0, 322.0, 276.0, 299.0, 274.0, 306.0, 271.0, 288.0, 287.0, 300.0, 276.0, 283.0, 308.0, 344.0, 425.0, 391.0, 423.0, 388.0, 418.0, 392.0, 419.0, 400.0, 445.0, 472.0, 447.0, 444.0, 410.0, 381.0, 457.0, 386.0, 412.0, 438.0, 454.0, 439.0, 424.0, 422.0, 400.0, 461.0, 391.0, 426.0, 459.0, 438.0, 414.0, 451.0, 411.0, 407.0, 419.0, 419.0, 414.0, 431.0, 463.0, 422.0, 451.0, 415.0, 418.0, 449.0, 431.0, 420.0, 454.0, 382.0, 421.0, 396.0, 433.0, 406.0, 406.0, 466.0, 419.0, 444.0, 413.0, 446.0, 412.0, 462.0, 448.0, 458.0, 480.0, 430.0, 407.0, 450.0, 429.0, 409.0, 446.0, 419.0, 415.0, 434.0, 411.0, 419.0, 492.0, 429.0, 446.0, 419.0, 460.0, 422.0, 428.0, 453.0, 460.0, 408.0, 455.0, 408.0, 472.0, 438.0, 450.0, 442.0, 403.0, 431.0, 438.0, 438.0, 431.0, 458.0, 493.0, 425.0, 437.0, 443.0, 473.0, 404.0, 448.0, 423.0, 389.0, 418.0, 433.0, 477.0, 422.0, 397.0, 463.0, 425.0, 428.0, 471.0, 407.0, 448.0, 520.0, 447.0, 413.0, 428.0, 420.0, 427.0, 461.0, 431.0, 399.0, 437.0, 466.0, 449.0, 410.0, 430.0, 445.0, 416.0, 393.0, 419.0, 428.0, 421.0, 406.0, 395.0, 476.0, 403.0, 442.0, 416.0, 418.0, 441.0, 475.0, 445.0, 417.0, 391.0, 419.0, 425.0, 432.0, 424.0, 432.0, 416.0, 424.0, 420.0, 442.0, 457.0, 410.0, 422.0, 442.0, 396.0, 453.0, 414.0, 422.0, 417.0, 437.0, 403.0, 426.0, 406.0, 428.0, 483.0, 414.0, 442.0, 450.0, 419.0, 443.0, 443.0, 434.0, 422.0, 429.0, 425.0, 415.0, 421.0, 415.0, 425.0, 414.0, 396.0, 438.0, 422.0, 393.0, 407.0, 442.0, 424.0, 318.0, 270.0, 288.0, 297.0, 279.0, 309.0, 284.0, 304.0, 298.0, 312.0, 275.0, 273.0, 297.0, 310.0, 265.0, 292.0, 281.0, 273.0, 288.0, 289.0, 296.0, 306.0, 287.0, 304.0, 324.0, 279.0, 291.0, 281.0, 287.0, 271.0, 286.0, 270.0, 280.0, 378.0, 453.0, 418.0, 413.0, 394.0, 436.0, 421.0, 445.0, 397.0, 402.0, 388.0, 415.0, 442.0, 409.0, 464.0, 463.0, 404.0, 446.0, 394.0, 447.0, 455.0, 435.0, 422.0, 427.0, 435.0, 423.0, 418.0, 410.0, 372.0, 391.0, 387.0, 396.0, 412.0, 407.0, 424.0, 422.0, 418.0, 418.0, 413.0, 392.0, 401.0, 450.0, 457.0, 443.0, 433.0, 410.0, 437.0, 452.0, 453.0, 457.0, 388.0, 439.0, 406.0, 433.0, 389.0, 433.0, 406.0, 444.0, 404.0, 397.0, 410.0, 441.0, 407.0, 426.0, 392.0, 413.0, 395.0, 444.0, 433.0, 411.0, 418.0, 449.0, 416.0, 419.0, 426.0, 440.0, 462.0, 450.0, 426.0, 484.0, 407.0, 415.0, 420.0, 436.0, 424.0, 408.0, 448.0, 382.0, 413.0, 440.0, 433.0, 444.0, 434.0, 408.0, 453.0, 430.0, 421.0, 441.0, 438.0, 401.0, 450.0, 460.0, 379.0, 405.0, 454.0, 404.0, 424.0, 411.0, 429.0, 422.0, 438.0, 399.0, 432.0, 468.0, 429.0, 423.0, 408.0, 410.0, 412.0, 436.0, 421.0, 433.0, 469.0, 389.0, 467.0, 431.0, 432.0, 415.0, 418.0, 449.0, 457.0, 416.0, 461.0, 398.0, 442.0, 435.0, 426.0, 445.0, 423.0, 403.0, 418.0, 421.0, 409.0, 412.0, 364.0, 413.0, 410.0, 443.0, 448.0, 389.0, 427.0, 443.0, 447.0, 440.0, 419.0, 468.0, 450.0, 402.0, 436.0, 389.0, 424.0, 411.0, 415.0, 396.0, 401.0, 363.0, 293.0, 275.0, 286.0, 283.0, 294.0, 291.0, 284.0, 291.0, 307.0, 289.0, 272.0, 274.0, 275.0, 296.0, 281.0, 290.0, 307.0, 297.0, 305.0, 297.0, 298.0, 292.0, 299.0, 317.0, 276.0, 292.0, 301.0, 312.0, 291.0, 291.0, 290.0, 302.0, 277.0, 280.0, 305.0, 311.0, 269.0, 272.0, 285.0, 308.0, 270.0, 278.0, 299.0, 267.0, 286.0, 276.0, 313.0, 276.0, 287.0, 295.0, 298.0, 291.0, 299.0, 280.0, 295.0, 269.0, 422.0, 422.0, 452.0, 427.0, 447.0, 441.0, 417.0, 453.0, 418.0, 422.0, 448.0, 436.0, 407.0, 426.0, 420.0, 398.0, 460.0, 426.0, 472.0, 421.0, 423.0, 431.0, 429.0, 444.0, 394.0, 401.0, 427.0, 390.0, 436.0, 398.0, 470.0, 348.0, 474.0, 447.0, 376.0, 465.0, 425.0, 372.0, 404.0, 392.0, 458.0, 446.0, 449.0, 435.0, 437.0, 463.0, 410.0, 449.0, 435.0, 442.0, 438.0, 432.0, 395.0, 459.0, 448.0, 461.0, 445.0, 434.0, 439.0, 505.0, 479.0, 474.0, 520.0, 440.0, 457.0, 447.0, 453.0, 497.0, 470.0, 452.0, 470.0, 471.0, 445.0, 469.0, 414.0, 469.0, 486.0, 445.0, 447.0, 479.0, 465.0, 466.0, 449.0, 461.0, 418.0, 435.0, 454.0, 451.0, 390.0, 428.0, 278.0, 281.0, 262.0, 294.0, 290.0, 300.0, 301.0, 305.0, 282.0, 286.0, 299.0, 306.0, 284.0, 265.0, 274.0, 296.0, 298.0, 270.0, 282.0, 400.0, 440.0, 434.0, 466.0, 377.0, 409.0, 423.0, 438.0, 409.0, 423.0, 464.0, 428.0, 408.0, 419.0, 456.0, 412.0, 404.0, 452.0, 466.0, 436.0, 463.0, 454.0, 437.0, 417.0, 432.0, 441.0, 415.0, 453.0, 396.0, 410.0, 434.0, 453.0, 406.0, 429.0, 431.0, 418.0, 407.0, 394.0, 392.0, 480.0, 420.0, 414.0, 422.0, 434.0, 391.0, 440.0, 477.0, 391.0, 444.0, 462.0, 447.0, 453.0, 414.0, 385.0, 416.0, 463.0, 460.0, 359.0, 301.0, 279.0, 281.0, 288.0, 281.0, 305.0, 261.0, 281.0, 285.0, 333.0, 467.0, 434.0, 464.0, 420.0, 375.0, 427.0, 456.0, 408.0, 472.0, 435.0, 439.0, 440.0, 435.0, 410.0, 433.0, 457.0, 463.0, 378.0, 394.0, 465.0, 397.0, 411.0, 418.0, 462.0, 454.0, 443.0, 452.0, 395.0, 438.0, 445.0, 405.0, 424.0, 400.0, 439.0, 415.0, 414.0, 440.0, 421.0, 358.0, 421.0, 373.0, 403.0, 441.0, 441.0, 398.0, 455.0, 411.0, 402.0, 413.0, 429.0, 387.0, 436.0, 417.0, 393.0, 463.0, 466.0, 397.0, 382.0, 443.0, 385.0, 435.0, 441.0, 418.0, 444.0, 424.0, 462.0, 437.0, 436.0, 422.0, 428.0, 434.0, 415.0, 453.0, 452.0, 417.0, 400.0, 401.0, 459.0, 452.0, 395.0, 399.0, 420.0, 410.0, 454.0, 414.0, 398.0, 450.0, 430.0, 398.0, 447.0, 436.0, 446.0, 385.0, 431.0, 409.0, 432.0, 441.0, 412.0, 425.0, 401.0, 411.0, 421.0, 402.0, 449.0, 432.0, 436.0, 424.0, 450.0, 434.0, 473.0, 471.0, 421.0, 460.0, 430.0, 441.0, 413.0, 399.0, 432.0, 396.0, 404.0, 430.0, 391.0, 399.0, 388.0, 445.0, 368.0, 420.0, 427.0, 418.0, 420.0, 382.0, 438.0, 418.0, 386.0, 463.0, 408.0, 397.0, 449.0, 402.0, 425.0, 401.0, 437.0, 398.0, 459.0, 392.0, 395.0, 488.0, 357.0, 396.0, 408.0, 392.0, 411.0, 422.0, 399.0, 413.0, 412.0, 427.0, 421.0, 403.0, 404.0, 447.0, 432.0, 404.0, 416.0, 414.0, 426.0, 428.0, 413.0, 422.0, 422.0, 415.0, 433.0, 415.0, 434.0, 407.0, 433.0, 406.0, 441.0, 457.0, 388.0, 390.0, 413.0, 428.0, 424.0, 429.0, 410.0, 391.0, 466.0, 446.0, 393.0, 409.0, 423.0, 430.0, 428.0, 443.0, 358.0, 278.0, 284.0, 286.0, 294.0, 292.0, 283.0, 279.0, 277.0, 279.0, 286.0, 290.0, 282.0, 312.0, 316.0, 300.0, 406.0, 420.0, 406.0, 470.0, 429.0, 430.0, 450.0, 435.0, 437.0, 392.0, 423.0, 455.0, 419.0, 428.0, 451.0, 416.0, 441.0, 412.0, 423.0, 431.0, 433.0, 421.0, 421.0, 445.0, 417.0, 398.0, 390.0, 408.0, 418.0, 426.0, 399.0, 390.0, 392.0, 395.0, 421.0, 408.0, 394.0, 393.0, 431.0, 407.0, 374.0, 447.0, 431.0, 448.0, 389.0, 442.0, 414.0, 412.0, 397.0, 442.0, 417.0, 420.0, 455.0, 416.0, 404.0, 391.0, 401.0, 415.0, 413.0, 429.0, 391.0, 419.0, 411.0, 413.0, 405.0, 414.0, 428.0, 409.0, 401.0, 388.0, 415.0, 389.0, 467.0, 428.0, 396.0, 426.0, 397.0, 454.0, 407.0, 439.0, 397.0, 430.0, 425.0, 405.0, 404.0, 453.0, 402.0, 424.0, 383.0, 378.0, 383.0, 407.0, 414.0, 425.0, 430.0, 414.0, 448.0, 417.0, 369.0, 417.0, 404.0, 424.0, 399.0, 424.0, 438.0, 429.0, 401.0, 409.0, 410.0, 396.0, 443.0, 429.0, 431.0, 454.0, 403.0, 419.0, 392.0, 380.0, 399.0, 441.0, 439.0, 477.0, 449.0, 514.0, 447.0, 425.0, 425.0, 446.0, 444.0, 430.0, 426.0, 488.0, 459.0, 430.0, 434.0, 391.0, 402.0, 405.0, 450.0, 411.0, 387.0, 432.0, 399.0, 456.0, 428.0, 411.0, 438.0, 417.0, 411.0, 441.0, 394.0, 413.0, 415.0, 413.0, 424.0, 411.0, 400.0, 397.0, 406.0, 433.0, 457.0, 440.0, 448.0, 391.0, 392.0, 414.0, 390.0, 439.0, 425.0, 392.0, 435.0, 399.0, 427.0, 419.0, 455.0, 388.0, 397.0, 463.0, 372.0, 449.0, 416.0, 404.0, 396.0]

非常感谢！

Answer 1

您可以在处理过程中添加进一步的步骤并检测卷积数据的局部最大值，而不仅仅是单个最大值点。

一种方法是使用scipy.signal.find_peaks

通过构建到the convolution method，您可以添加find_peaks 步骤，如下所示：

import numpy as np
from scipy import signal
from matplotlib import pyplot as plt

d = [ your data ]

# Convolution part
dary = np.array(d)
dary -= np.average(dary)

step = np.hstack((np.ones(len(dary)), -1*np.ones(len(dary))))

dary_step = np.convolve(dary, step, mode='valid')

# Get the peaks of the convolution
peaks = signal.find_peaks(dary_step, width=20)[0]

# plots
plt.figure()

plt.plot(dary)

plt.plot(dary_step/10)

for ii in range(len(peaks)):
    plt.plot((peaks[ii], peaks[ii]), (-1500, 1500), 'r')

plt.show()

这会渲染输出：

上面的方法找到向上的台阶，而你可以通过简单地使用scipy.signal.find_peaks(-dary_step)来找到向下的台阶——正如cmets中所建议的那样。